Firebase is back at Google I/O on May 20-21! Register now.

vertexai package
Stay organized with collections Save and categorize content based on your preferences.

The Vertex AI in Firebase Web SDK.

Functions

Function	Description
function(app, ...)
getVertexAI(app, options)	Returns a VertexAI instance for the given app.
function(vertexAI, ...)
getGenerativeModel(vertexAI, modelParams, requestOptions)	Returns a GenerativeModel class with methods for inference and other functionality.
getImagenModel(vertexAI, modelParams, requestOptions)	*(Public Preview)* Returns an ImagenModel class with methods for using Imagen.Only Imagen 3 models (named `imagen-3.0-*`) are supported.

Classes

Class	Description
ArraySchema	Schema class for "array" types. The `items` param should refer to the type of item that can be a member of the array.
BooleanSchema	Schema class for "boolean" types.
ChatSession	ChatSession class that enables sending chat messages and stores history of sent and received messages so far.
GenerativeModel	Class for generative model APIs.
ImagenImageFormat	*(Public Preview)* Defines the image format for images generated by Imagen.Use this class to specify the desired format (JPEG or PNG) and compression quality for images generated by Imagen. This is typically included as part of ImagenModelParams.
ImagenModel	*(Public Preview)* Class for Imagen model APIs.This class provides methods for generating images using the Imagen model.
IntegerSchema	Schema class for "integer" types.
NumberSchema	Schema class for "number" types.
ObjectSchema	Schema class for "object" types. The `properties` param must be a map of `Schema` objects.
Schema	Parent class encompassing all Schema types, with static methods that allow building specific Schema types. This class can be converted with `JSON.stringify()` into a JSON string accepted by Vertex AI REST endpoints. (This string conversion is automatically done when calling SDK methods.)
StringSchema	Schema class for "string" types. Can be used with or without enum values.
VertexAIError	Error class for the Vertex AI in Firebase SDK.
VertexAIModel	Base class for Vertex AI in Firebase model APIs.

Enumerations

Enumeration	Description
BlockReason	Reason that a prompt was blocked.
FinishReason	Reason that a candidate finished.
FunctionCallingMode
HarmBlockMethod
HarmBlockThreshold	Threshold above which a prompt or candidate will be blocked.
HarmCategory	Harm categories that would cause prompts or candidates to be blocked.
HarmProbability	Probability that a prompt or candidate matches a harm category.
HarmSeverity	Harm severity levels.
ImagenAspectRatio	*(Public Preview)* Aspect ratios for Imagen images.To specify an aspect ratio for generated images, set the `aspectRatio` property in your ImagenGenerationConfig.See the the documentation for more details and examples of the supported aspect ratios.
ImagenPersonFilterLevel	*(Public Preview)* A filter level controlling whether generation of images containing people or faces is allowed.See the personGeneration documentation for more details.
ImagenSafetyFilterLevel	*(Public Preview)* A filter level controlling how aggressively to filter sensitive content.Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI are assessed against a list of safety filters, which include 'harmful categories' (for example, `violence`, `sexual`, `derogatory`, and `toxic`). This filter level controls how aggressively to filter out potentially harmful content from responses. See the documentation and the Responsible AI and usage guidelines for more details.
Modality	Content part modality.
SchemaType	Contains the list of OpenAPI data types as defined by the OpenAPI specification
VertexAIErrorCode	Standardized error codes that VertexAIError can have.

Interfaces

Interface	Description
BaseParams	Base parameters for a number of methods.
Citation	A single citation.
CitationMetadata	Citation metadata that may be found on a GenerateContentCandidate.
Content	Content type for both prompts and response candidates.
CountTokensRequest	Params for calling GenerativeModel.countTokens()
CountTokensResponse	Response from calling GenerativeModel.countTokens().
CustomErrorData	Details object that contains data originating from a bad HTTP response.
Date_2	Protobuf google.type.Date
EnhancedGenerateContentResponse	Response object wrapped with helper methods.
ErrorDetails	Details object that may be included in an error response.
FileData	Data pointing to a file uploaded on Google Cloud Storage.
FileDataPart	Content part interface if the part represents FileData
FunctionCall	A predicted FunctionCall returned from the model that contains a string representing the FunctionDeclaration.name and a structured JSON object containing the parameters and their values.
FunctionCallingConfig
FunctionCallPart	Content part interface if the part represents a FunctionCall.
FunctionDeclaration	Structured representation of a function declaration as defined by the OpenAPI 3.0 specification. Included in this declaration are the function name and parameters. This `FunctionDeclaration` is a representation of a block of code that can be used as a Tool by the model and executed by the client.
FunctionDeclarationsTool	A `FunctionDeclarationsTool` is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model.
FunctionResponse	The result output from a FunctionCall that contains a string representing the FunctionDeclaration.name and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a FunctionCall made based on model prediction.
FunctionResponsePart	Content part interface if the part represents FunctionResponse.
GenerateContentCandidate	A candidate returned as part of a GenerateContentResponse.
GenerateContentRequest	Request sent through GenerativeModel.generateContent()
GenerateContentResponse	Individual response from GenerativeModel.generateContent() and GenerativeModel.generateContentStream(). `generateContentStream()` will return one in each chunk until the stream is done.
GenerateContentResult	Result object returned from GenerativeModel.generateContent() call.
GenerateContentStreamResult	Result object returned from GenerativeModel.generateContentStream() call. Iterate over `stream` to get chunks as they come in and/or use the `response` promise to get the aggregated response when the stream is done.
GenerationConfig	Config options for content-related requests
GenerativeContentBlob	Interface for sending an image.
GroundingAttribution
GroundingMetadata	Metadata returned to client when grounding is enabled.
ImagenGCSImage	An image generated by Imagen, stored in a Cloud Storage for Firebase bucket.This feature is not available yet.
ImagenGenerationConfig	*(Public Preview)* Configuration options for generating images with Imagen.See the documentation for more details.
ImagenGenerationResponse	*(Public Preview)* The response from a request to generate images with Imagen.
ImagenInlineImage	*(Public Preview)* An image generated by Imagen, represented as inline data.
ImagenModelParams	*(Public Preview)* Parameters for configuring an ImagenModel.
ImagenSafetySettings	*(Public Preview)* Settings for controlling the aggressiveness of filtering out sensitive content.See the documentation for more details.
InlineDataPart	Content part interface if the part represents an image.
ModalityTokenCount	Represents token counting info for a single modality.
ModelParams	Params passed to getGenerativeModel().
ObjectSchemaInterface	Interface for ObjectSchema class.
PromptFeedback	If the prompt was blocked, this will be populated with `blockReason` and the relevant `safetyRatings`.
RequestOptions	Params passed to getGenerativeModel().
RetrievedContextAttribution
SafetyRating	A safety rating associated with a GenerateContentCandidate
SafetySetting	Safety setting that can be sent as part of request parameters.
SchemaInterface	Interface for Schema class.
SchemaParams	Params passed to Schema static methods to create specific Schema classes.
SchemaRequest	Final format for Schema params passed to backend requests.
SchemaShared	Basic Schema properties shared across several Schema-related types.
Segment
StartChatParams	Params for GenerativeModel.startChat().
TextPart	Content part interface if the part represents a text string.
ToolConfig	Tool config. This config is shared for all tools provided in the request.
UsageMetadata	Usage metadata about a GenerateContentResponse.
VertexAI	An instance of the Vertex AI in Firebase SDK.
VertexAIOptions	Options when initializing the Vertex AI in Firebase SDK.
VideoMetadata	Describes the input video content.
WebAttribution

Variables

Variable	Description
POSSIBLE_ROLES	Possible roles.

Type Aliases

Type Alias	Description
Part	Content part - includes text, image/video, or function call/response part types.
Role	Role is the producer of the content.
Tool	Defines a tool that model can call to access external knowledge.
TypedSchema	A type that includes all specific Schema types.

function(app, ...)

getVertexAI(app, options)

Returns a VertexAI instance for the given app.

Signature:

exportdeclarefunctiongetVertexAI(app?:FirebaseApp,options?:VertexAIOptions):VertexAI;

Parameters

Parameter	Type	Description
app	FirebaseApp	The FirebaseApp to use.
options	VertexAIOptions

Returns:

VertexAI

function(vertexAI, ...)

getGenerativeModel(vertexAI, modelParams, requestOptions)

Returns a GenerativeModel class with methods for inference and other functionality.

Signature:

exportdeclarefunctiongetGenerativeModel(vertexAI:VertexAI,modelParams:ModelParams,requestOptions?:RequestOptions):GenerativeModel;

Parameters

Parameter	Type	Description
vertexAI	VertexAI
modelParams	ModelParams
requestOptions	RequestOptions

Returns:

GenerativeModel

getImagenModel(vertexAI, modelParams, requestOptions)

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

Returns an ImagenModel class with methods for using Imagen.

Only Imagen 3 models (named imagen-3.0-*) are supported.

Signature:

exportdeclarefunctiongetImagenModel(vertexAI:VertexAI,modelParams:ImagenModelParams,requestOptions?:RequestOptions):ImagenModel;

Parameters

Parameter	Type	Description
vertexAI	VertexAI	An instance of the Vertex AI in Firebase SDK.
modelParams	ImagenModelParams	Parameters to use when making Imagen requests.
requestOptions	RequestOptions	Additional options to use when making requests.

Returns:

ImagenModel

Exceptions

If the apiKey or projectId fields are missing in your Firebase config.

POSSIBLE_ROLES

Possible roles.

Signature:

POSSIBLE_ROLES:readonly["user","model","function","system"]

Part

Content part - includes text, image/video, or function call/response part types.

Signature:

exporttypePart=TextPart|InlineDataPart|FunctionCallPart|FunctionResponsePart|FileDataPart;

Role

Role is the producer of the content.

Signature:

exporttypeRole=(typeofPOSSIBLE_ROLES)[number];

Tool

Defines a tool that model can call to access external knowledge.

Signature:

exportdeclaretypeTool=FunctionDeclarationsTool;

TypedSchema

A type that includes all specific Schema types.

Signature:

exporttypeTypedSchema=IntegerSchema|NumberSchema|StringSchema|BooleanSchema|ObjectSchema|ArraySchema;

BlockReason

Reason that a prompt was blocked.

Signature:

exportdeclareenumBlockReason

Enumeration Members

Member	Value	Description
BLOCKLIST	`"BLOCKLIST"`	Content was blocked because it contained terms from the terminology blocklist.
OTHER	`"OTHER"`	Content was blocked, but the reason is uncategorized.
PROHIBITED_CONTENT	`"PROHIBITED_CONTENT"`	Content was blocked due to prohibited content.
SAFETY	`"SAFETY"`	Content was blocked by safety settings.

FinishReason

Reason that a candidate finished.

Signature:

exportdeclareenumFinishReason

Enumeration Members

Member	Value	Description
BLOCKLIST	`"BLOCKLIST"`	The candidate content contained forbidden terms.
MALFORMED_FUNCTION_CALL	`"MALFORMED_FUNCTION_CALL"`	The function call generated by the model was invalid.
MAX_TOKENS	`"MAX_TOKENS"`	The maximum number of tokens as specified in the request was reached.
OTHER	`"OTHER"`	Unknown reason.
PROHIBITED_CONTENT	`"PROHIBITED_CONTENT"`	The candidate content potentially contained prohibited content.
RECITATION	`"RECITATION"`	The candidate content was flagged for recitation reasons.
SAFETY	`"SAFETY"`	The candidate content was flagged for safety reasons.
SPII	`"SPII"`	The candidate content potentially contained Sensitive Personally Identifiable Information (SPII).
STOP	`"STOP"`	Natural stop point of the model or provided stop sequence.

FunctionCallingMode

Signature:

exportdeclareenumFunctionCallingMode

Enumeration Members

Member	Value	Description
ANY	`"ANY"`	Model is constrained to always predicting a function call only. If `allowed_function_names` is set, the predicted function call will be limited to any one of `allowed_function_names`, else the predicted function call will be any one of the provided `function_declarations`.
AUTO	`"AUTO"`	Default model behavior; model decides to predict either a function call or a natural language response.
NONE	`"NONE"`	Model will not predict any function call. Model behavior is same as when not passing any function declarations.

HarmBlockMethod

Signature:

exportdeclareenumHarmBlockMethod

Enumeration Members

Member	Value	Description
PROBABILITY	`"PROBABILITY"`	The harm block method uses the probability score.
SEVERITY	`"SEVERITY"`	The harm block method uses both probability and severity scores.

HarmBlockThreshold

Threshold above which a prompt or candidate will be blocked.

Signature:

exportdeclareenumHarmBlockThreshold

Enumeration Members

Member	Value	Description
BLOCK_LOW_AND_ABOVE	`"BLOCK_LOW_AND_ABOVE"`	Content with `NEGLIGIBLE` will be allowed.
BLOCK_MEDIUM_AND_ABOVE	`"BLOCK_MEDIUM_AND_ABOVE"`	Content with `NEGLIGIBLE` and `LOW` will be allowed.
BLOCK_NONE	`"BLOCK_NONE"`	All content will be allowed.
BLOCK_ONLY_HIGH	`"BLOCK_ONLY_HIGH"`	Content with `NEGLIGIBLE`, `LOW`, and `MEDIUM` will be allowed.

HarmCategory

Harm categories that would cause prompts or candidates to be blocked.

Signature:

exportdeclareenumHarmCategory

Enumeration Members

Member	Value	Description
HARM_CATEGORY_DANGEROUS_CONTENT	`"HARM_CATEGORY_DANGEROUS_CONTENT"`
HARM_CATEGORY_HARASSMENT	`"HARM_CATEGORY_HARASSMENT"`
HARM_CATEGORY_HATE_SPEECH	`"HARM_CATEGORY_HATE_SPEECH"`
HARM_CATEGORY_SEXUALLY_EXPLICIT	`"HARM_CATEGORY_SEXUALLY_EXPLICIT"`

HarmProbability

Probability that a prompt or candidate matches a harm category.

Signature:

exportdeclareenumHarmProbability

Enumeration Members

Member	Value	Description
HIGH	`"HIGH"`	Content has a high chance of being unsafe.
LOW	`"LOW"`	Content has a low chance of being unsafe.
MEDIUM	`"MEDIUM"`	Content has a medium chance of being unsafe.
NEGLIGIBLE	`"NEGLIGIBLE"`	Content has a negligible chance of being unsafe.

HarmSeverity

Harm severity levels.

Signature:

exportdeclareenumHarmSeverity

Enumeration Members

Member	Value	Description
HARM_SEVERITY_HIGH	`"HARM_SEVERITY_HIGH"`	High level of harm severity.
HARM_SEVERITY_LOW	`"HARM_SEVERITY_LOW"`	Low level of harm severity.
HARM_SEVERITY_MEDIUM	`"HARM_SEVERITY_MEDIUM"`	Medium level of harm severity.
HARM_SEVERITY_NEGLIGIBLE	`"HARM_SEVERITY_NEGLIGIBLE"`	Negligible level of harm severity.

ImagenAspectRatio

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

Aspect ratios for Imagen images.

To specify an aspect ratio for generated images, set the aspectRatio property in your ImagenGenerationConfig.

See the the documentation for more details and examples of the supported aspect ratios.

Signature:

exportdeclareenumImagenAspectRatio

Enumeration Members

Member	Value	Description
LANDSCAPE_16x9	`"16:9"`	*(Public Preview)* Landscape (16:9) aspect ratio.
LANDSCAPE_3x4	`"3:4"`	*(Public Preview)* Landscape (3:4) aspect ratio.
PORTRAIT_4x3	`"4:3"`	*(Public Preview)* Portrait (4:3) aspect ratio.
PORTRAIT_9x16	`"9:16"`	*(Public Preview)* Portrait (9:16) aspect ratio.
SQUARE	`"1:1"`	*(Public Preview)* Square (1:1) aspect ratio.

ImagenPersonFilterLevel

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

A filter level controlling whether generation of images containing people or faces is allowed.

See the personGeneration documentation for more details.

Signature:

exportdeclareenumImagenPersonFilterLevel

Enumeration Members

Member	Value	Description
ALLOW_ADULT	`"allow_adult"`	*(Public Preview)* Allow generation of images containing adults only; images of children are filtered out.Generation of images containing people or faces may require your use case to be reviewed and approved by Cloud support; see the Responsible AI and usage guidelines for more details.
ALLOW_ALL	`"allow_all"`	*(Public Preview)* Allow generation of images containing adults only; images of children are filtered out.Generation of images containing people or faces may require your use case to be reviewed and approved by Cloud support; see the Responsible AI and usage guidelines for more details.
BLOCK_ALL	`"dont_allow"`	*(Public Preview)* Disallow generation of images containing people or faces; images of people are filtered out.

ImagenSafetyFilterLevel

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

A filter level controlling how aggressively to filter sensitive content.

Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI are assessed against a list of safety filters, which include 'harmful categories' (for example, violence, sexual, derogatory, and toxic). This filter level controls how aggressively to filter out potentially harmful content from responses. See the documentation and the Responsible AI and usage guidelines for more details.

Signature:

exportdeclareenumImagenSafetyFilterLevel

Enumeration Members

Member	Value	Description
BLOCK_LOW_AND_ABOVE	`"block_low_and_above"`	*(Public Preview)* The most aggressive filtering level; most strict blocking.
BLOCK_MEDIUM_AND_ABOVE	`"block_medium_and_above"`	*(Public Preview)* Blocks some sensitive prompts and responses.
BLOCK_NONE	`"block_none"`	*(Public Preview)* The least aggressive filtering level; blocks very few sensitive prompts and responses.Access to this feature is restricted and may require your case to be reviewed and approved by Cloud support.
BLOCK_ONLY_HIGH	`"block_only_high"`	*(Public Preview)* Blocks few sensitive prompts and responses.

Modality

Content part modality.

Signature:

exportdeclareenumModality

Enumeration Members

Member	Value	Description
AUDIO	`"AUDIO"`	Audio.
DOCUMENT	`"DOCUMENT"`	Document (for example, PDF).
IMAGE	`"IMAGE"`	Image.
MODALITY_UNSPECIFIED	`"MODALITY_UNSPECIFIED"`	Unspecified modality.
TEXT	`"TEXT"`	Plain text.
VIDEO	`"VIDEO"`	Video.

SchemaType

Contains the list of OpenAPI data types as defined by the OpenAPI specification

Signature:

exportdeclareenumSchemaType

Enumeration Members

Member	Value	Description
ARRAY	`"array"`	Array type.
BOOLEAN	`"boolean"`	Boolean type.
INTEGER	`"integer"`	Integer type.
NUMBER	`"number"`	Number type.
OBJECT	`"object"`	Object type.
STRING	`"string"`	String type.

VertexAIErrorCode

Standardized error codes that VertexAIError can have.

Signature:

exportdeclareconstenumVertexAIErrorCode

Enumeration Members

Member	Value	Description
API_NOT_ENABLED	`"api-not-enabled"`	An error due to the Firebase API not being enabled in the Console.
ERROR	`"error"`	A generic error occurred.
FETCH_ERROR	`"fetch-error"`	An error occurred while performing a fetch.
INVALID_CONTENT	`"invalid-content"`	An error associated with a Content object.
INVALID_SCHEMA	`"invalid-schema"`	An error due to invalid Schema input.
NO_API_KEY	`"no-api-key"`	An error occurred due to a missing Firebase API key.
NO_APP_ID	`"no-app-id"`	An error occured due to a missing Firebase app ID.
NO_MODEL	`"no-model"`	An error occurred due to a model name not being specified during initialization.
NO_PROJECT_ID	`"no-project-id"`	An error occurred due to a missing project ID.
PARSE_FAILED	`"parse-failed"`	An error occurred while parsing.
REQUEST_ERROR	`"request-error"`	An error occurred in a request.
RESPONSE_ERROR	`"response-error"`	An error occurred in a response.

vertexai package Stay organized with collections Save and categorize content based on your preferences.

Functions

Classes

Enumerations

Interfaces

Variables

Type Aliases

function(app, ...)

getVertexAI(app, options)

Parameters

function(vertexAI, ...)

getGenerativeModel(vertexAI, modelParams, requestOptions)

Parameters

getImagenModel(vertexAI, modelParams, requestOptions)

Parameters

Exceptions

POSSIBLE_ROLES

Part

Role

Tool

TypedSchema

BlockReason

Enumeration Members

FinishReason

Enumeration Members

FunctionCallingMode

Enumeration Members

HarmBlockMethod

Enumeration Members

HarmBlockThreshold

Enumeration Members

HarmCategory

Enumeration Members

HarmProbability

Enumeration Members

HarmSeverity

Enumeration Members

ImagenAspectRatio

Enumeration Members

ImagenPersonFilterLevel

Enumeration Members

ImagenSafetyFilterLevel

Enumeration Members

Modality

Enumeration Members

SchemaType

Enumeration Members

VertexAIErrorCode

Enumeration Members

vertexai package
Stay organized with collections Save and categorize content based on your preferences.