Migrate from 0.9 to 1.0

Genkit 1.0 introduces many feature enhancements that improve overall functionality; it also has some breaking changes. If you have been developing applications with Genkit 0.9, you need to update your application code when you upgrade to the latest version of Genkit. This guide outlines the most significant changes, and explains how to migrate your existing applications smoothly.

Beta APIs

We're introducing an unstable, Beta API channel, and leaving session, chat and Genkit client APIs in beta as we continue to refine them. More specifically, the following functions are currently in the beta namespace:

  • ai.chat
  • ai.createSession
  • ai.loadSession
  • ai.currentSession
  • ai.defineFormat
  • ai.defineInterrupt

Old:

import{genkit}from'genkit';constai=genkit({...})constsession=ai.createSession({...})

New:

import{genkit}from'genkit/beta';constai=genkit({...})constsession=ai.createSession({...})

Old:

import{runFlow,streamFlow}from'genkit/client';

New:

import{runFlow,streamFlow}from'genkit/beta/client';

Introducing new @genkit-ai/express package

This new package contains utilities to make it easier to build an Express.js server with Genkit. You can find more details about this on this page.

startFlowServer has moved from part of the genkit object to this new @genkit-ai/express package; to use startFlowServer, you must update your imports.

Old:

constai=genkit({...});ai.startFlowServer({flows:[myFlow1,myFlow2],});

New:

import{startFlowServer}from'@genkit-ai/express';startFlowServer({flows:[myFlow1,myFlow2],});

Changes to Flows

There are several changes to flows in 1.0:

  • ai.defineStreamingFlow has been consolidated into ai.defineFlow,
  • onFlow has been replaced by onCallGenkit,
  • run has moved to ai.run,
  • There are changes to working with auth.

The run function for custom trace blocks has moved to part of the genkit object; use ai.run to invoke it instead.

Old:

ai.defineFlow({name:'banana'},async(input)=>{conststep=awaitrun('myCode',async()=>{return'something'});})

New:

ai.defineFlow({name:'banana'},async(input)=>{conststep=awaitai.run('myCode',async()=>{return'something'});})

ai.defineStreamingFlow has been removed; use ai.defineFlow instead. Also, streamingCallback has moved to a field inside the second argument of the flow function and is now called sendChunk.

Old:

constflow=ai.defineStreamingFlow({name:'banana'},async(input,streamingCallback)=>{streamingCallback({chunk:1});})const{stream}=awaitflow()forawait(constchunkofstream){// ...}

New:

constflow=ai.defineFlow({name:'banana'},async(input,{context,sendChunk})=>{sendChunk({chunk:1});})const{stream,output}=flow.stream(input);forawait(constchunkofstream){// ...}

FlowAuth auth is now called context. You can access auth as a field inside context:

Old:

ai.defineFlow({name:'banana'},async(input)=>{constauth=getFlowAuth();// ...})

New:

ai.defineFlow({name:'banana'},async(input,{context})=>{constauth=context.auth;})

onFlow moved to firebase-functions/https package and has been renamed to onCallGenkit. The following snippet shows an example of how to use it.

Old

import{onFlow}from"@genkit-ai/firebase/functions";exportconstgeneratePoem=onFlow(ai,{name:"jokeTeller",inputSchema:z.string().nullable(),outputSchema:z.string(),streamSchema:z.string(),},async(type,streamingCallback)=>{const{stream,response}=awaitai.generateStream(`Tell me a longish ${type??"dad"} joke.`);forawait(constchunkofstream){streamingCallback(chunk.text);}return(awaitresponse).text;});

New:

import{onCallGenkit}from"firebase-functions/https";import{defineSecret}from"firebase-functions/params";import{genkit,z}from"genkit";constapiKey=defineSecret("GEMINI_API_KEY");constai=genkit({plugins:[googleAI()],model:gemini15Flash,});exportconstjokeTeller=ai.defineFlow({name:"jokeTeller",inputSchema:z.string().nullable(),outputSchema:z.string(),streamSchema:z.string(),},async(type,{sendChunk})=>{const{stream,response}=ai.generateStream(`Tell me a longish ${type??"dad"} joke.`);forawait(constchunkofstream){sendChunk(chunk.text);}return(awaitresponse).text;});exportconsttellJoke=onCallGenkit({secrets:[apiKey]},jokeTeller);

Auth policies have been removed from defineFlow. Handling of auth policies is now server-dependent.

Old:

exportconstsimpleFlow=ai.defineFlow({name:'simpleFlow',authPolicy:(auth,input)=>{// auth policy},},async(input)=>{// Flow logic here...});

The following snippet shows an example of handling auth in Express.

New:

import{UserFacingError}from'genkit';import{ContextProvider,RequestData}from'genkit/context';import{expressHandler,startFlowServer}from'@genkit-ai/express';constcontext:ContextProvider<Context>=(req:RequestData)=>{return{auth:parseAuthToken(req.headers['authorization']),};};exportconstsimpleFlow=ai.defineFlow({name:'simpleFlow',},async(input,{context})=>{if(!context.auth){thrownewUserFacingError("UNAUTHORIZED","Authorization required.");}if(input.uid!==context.auth.uid){thrownewUserFacingError("UNAUTHORIZED","You may only summarize your own profile data.");}// Flow logic here...});constapp=express();app.use(express.json());app.post('/simpleFlow',expressHandler(simpleFlow,{context}));app.listen(8080);// orstartFlowServer(flows:[withContextProvider(simpleFlow,context)],port:8080);

For more details, refer to the auth documentation.

The following snippet shows an example of handling auth in Cloud Functions for Firebase:

import{genkit}from'genkit';import{onCallGenkit}from'firebase-functions/https';constai=genkit({...});;constsimpleFlow=ai.defineFlow({name:'simpleFlow',},async(input)=>{// Flow logic here...});exportconstselfSummary=onCallGenkit({authPolicy:(auth,data)=>auth?.token?.['email_verified'] && auth?.token?.['admin'],},simpleFlow);

Prompts

We've made several changes and improvements to prompts.

You can define separate templates for prompt and system messages:

consthello=ai.definePrompt({name:'hello',system:'talk like a pirate.',prompt:'hello {{ name }}',input:{schema:z.object({name:z.string()})}});const{text}=awaithello({name:'Genkit'});

Alternatively, you can define multi-message prompts in the messages field:

consthello=ai.definePrompt({name:'hello',messages:'{{ role "system" }} talk like a pirate. {{ role "user" }} hello {{ name }}',input:{schema:z.object({name:z.string()})}});

Instead of prompt templates you can use a function:

ai.definePrompt({name:'hello',prompt:async(input,{context})=>{return`hello ${input.name}`},input:{schema:z.object({name:z.string()})}});

You can access the context (including auth information) from within the prompt:

consthello=ai.definePrompt({name:'hello',messages:'hello {{ @auth.email }}',});

Streaming functions do not require an await

Old:

const{stream,response}=awaitai.generateStream(`hi`);const{stream,output}=awaitmyflow.stream(`hi`);

New:

const{stream,response}=ai.generateStream(`hi`);const{stream,output}=myflow.stream(`hi`);

Embed has a new return type

We've added support for multimodal embeddings. Instead of returning just a single embedding vector, Embed returns an array of embedding objects, each containing an embedding vector and metadata.

Old:

constresponse=awaitai.embed({embedder,content,options});// returns number[]

New:

constresponse=awaitai.embed({embedder,content,options});// returns Embedding[]constfirstEmbeddingVector=response[0].embedding;// is number[]