Build the generate() API from source

• Pre-requisites
• Clone the onnxruntime-genai repo
• Build the generate() API
• Install the library into your application

Pre-requisites

• cmake
• .NET6 (if building C#)

Clone the onnxruntime-genai repo

git clone https://github.com/microsoft/onnxruntime-genai cd onnxruntime-genai 

Build the generate() API

This step assumes that you are in the root of the onnxruntime-genai repo.

All of the build commands below have a --config argument, which takes the following options:

• Release builds release binaries
• Debug build binaries with debug symbols
• RelWithDebInfo builds release binaries with debug info

Build Python API

Windows CPU build

python build.py --config Release 

Windows DirectML build

python build.py --use_dml--config Release 

Linux build

python build.py --config Release 

Linux CUDA build

python build.py --use_cuda--config Release 

Mac build

python build.py --config Release 

Build Java API

python build.py --build_java--config Release 

Build for Android

If building on Windows, install ninja.

pip install ninja 

Run the build script.

python build.py --build_java--android--android_home <path to your Android SDK> --android_ndk_path <path to your NDK installation> --android_abi[armeabi-v7a|arm64-v8a|x86|x86_64] --config Release 

Install the library into your application

Install Python wheel

# Change dir to the folder containing the onnxruntime-genai wheel# Example for Linux: cd build/Linux/Release/wheel/ pip install*.whl 

Install NuGet

Coming soon

Install JAR

Copy build/Windows/Release/src/java/build/libs/*.jar into your application.

Install AAR

Copy build/Android/Release/src/java/build/android/outputs/aar/onnxruntime-genai-release.aar into your application.

Install C/C++ header file and library

Windows

Use the header in src\ort_genai.h and the libraries in build\Windows\Release

Linux

Use the header in src/ort_genai.h and the libraries in build/Linux/Release