Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Windows Copilot Runtime provides a variety of artificial intelligence (AI) features through Windows AI APIs and Windows machine learning (ML). The Windows AI APIs enable AI capabilities without the need to find, run, or optimize your own machine learning (ML) model. The models that power Windows Copilot Runtime on Copilot+ PCs run locally and continuously in the background.
See Windows Copilot Runtime Sample for a complete app that demonstrates how to use the Windows Copilot Runtime with WinUI.
Build your first AI-powered Windows app
To build your first Windows app with Visual Studio and some simple Windows AI APIs, just meet the prerequisites and use the provided example code in Get started building an app with Windows AI APIs.
From there, you can jump into short tutorials that build an app leveraging specific Windows Copilot Runtime APIs such as the Phi Silica walthrough, Imaging walthrough and OCR walthrough.
Try the APIs and models on your PC
AI Dev Gallery is a demo app—available from the Microsoft Store—that lets you quickly download, try out, and use Windows AI APIs and models.
In AI Dev Gallery, select the WCR API tab menu item, then select the Phi Silica sample. If the model is already available on your device, then that sample will run straight away. Otherwise, select Request model to download the model. Once downloaded, that sample will be activated. Learn more about the AI Dev Gallery in What is the AI Dev Gallery?.
Overview of available APIs
Here are a few ready-to-use AI features that you can tap into from your Windows app:
- Phi Silica. A local, ready-to-use language model. See Get started with Phi Silica.
- AI text recognition. Recognize text in images, and convert images/pdfs into searchable text. See Get started with AI text recognition.
- AI Imaging. Scale and sharpen images using AI (Image Super Resolution), as well as identify objects within an image (Image Segmentation). See Get Started with AI imaging.
- Windows Studio Effects. Apply AI effects to your device's device's built-in camera and microphone. See Windows Studio Effects Overview (Preview).
Phi Silica
Similar to OpenAI's GPT Large Language Model (LLM), which powers ChatGPT, Phi is a Small Language Model (SLM) developed by Microsoft Research to perform language-processing tasks on a local device. Phi Silica is specifically designed for Windows devices that have a Neural Processing Unit (NPU), allowing text generation and conversation features to run in a high performance, hardware-accelerated way directly on the device. Phi Silica is not available in China.
Also see Get started with Phi Silica.
Tip
To improve accessibility and readability, this page uses still images in the default view. You can click an image to see the animated version.
Text recognition
The text recognition APIs enable the recognition of text in an image, and the conversion on a local device of different types of documents (such as scanned paper documents, PDF files, and images captured by a digital camera) into editable and searchable data.
Also see Get started with AI text recognition
Image Super Resolution
The Image Super Resolution APIs enable image sharpening and scaling.
Also see What can I do with Image Super Resolution?.
Image Segmentation
The Image Segmentation APIs enable segmentation of images.
Also see What can I do with Image Segmentation?.
Image Description
The Image Description APIs describes images in natural language. Image Description features are not available in China.
Also see Get text description from an image
Object Erase
The Object Erase APIs allows for removing objects from images.
Also see Get started with Object Erase
Additional AI features
Windows Studio Effects. Windows devices that have compatible Neural Processing Units (NPUs) integrate Windows Studio Effects into the device's built-in camera and microphone settings. You can apply special effects that use AI, including: Background Blur, Eye Contact correction, Automatic Framing, Portrait Light correction, Creative Filters, and Voice Focus for filtering out background noise. See Windows Studio Effects Overview (Preview).
Recall(Not currently supported as an API). Enables users to quickly find artifacts from their past activity, such as documents, images, websites, and more. As a developer, you can enrich your users' Recall experience with their app by adding contextual information to the underlying vector database by using the User Activity API. That integration will help your users pick up where they left off in your app, thereby improving app engagement and users' seamless flow between Windows and your app. See Recall overview.
Live Caption Translations (Not yet supported). Help everyone using Windows—including those who are deaf or hard of hearing—better understand audio by viewing captions of spoken content (even when the audio content is in a language that's different from the system's preferred language).
Content moderation
Learn how Windows Copilot Runtime moderates content, and how to adjust sensitivity filters. See Content safety moderation with Windows Copilot Runtime.
When utilizing AI features, we recommend that you review: Developing Responsible Generative AI Applications and Features on Windows.
Additional resources
- Code samples and tutorials. A collection of samples that demonstrate a variety of ways to use AI to enhance your Windows apps.
- Integrate AI in enterprise apps using Windows Copilot Runtime APIs. Watch the demo session from the November 2024 Microsoft Ignite conference.
- Provide feedback on these APIs and their functionality by creating a new Issue in the Windows App SDK GitHub repo or by responding to an existing issue.
