Sharing tabs, windows, and screens is already possible on the web platform with the Screen Capture API. When a web app calls getDisplayMedia(), Chrome prompts the user to share a tab, window, or screen with the web app as a MediaStreamTrack video.
Many web apps that use getDisplayMedia() show the user a video preview of the captured surface. For example, video conferencing apps will often stream this video to remote users while also rendering it to a local HTMLVideoElement, so that the local user would constantly see a preview of what they're sharing.
This documentation introduces the new Captured Surface Control API in Chrome, which lets your web app scroll a captured tab, as well as read and write the zoom level of a captured tab.
Why use Captured Surface Control?
All video conferencing apps suffer from the same drawback. If the user wishes to interact with a captured tab or window, the user must switch to that surface, taking them away from the video conferencing app. This presents some challenges:
- The user can't see the captured app and the video feeds of remote users at the same time unless they use Picture-in-Picture or separate side-by-side windows for the video conference tab and the shared tab. On a smaller screen, this could be difficult.
- The user is burdened by the need to jump between the video conferencing app and the captured surface.
- The user loses access to the controls exposed by the video conferencing app while they are away from it; for example, an embedded chat app, emoji reactions, notifications about users asking to join the call, multimedia and layout controls, and other useful video conferencing features.
- The presenter cannot delegate control to remote participants. This leads to the all too familiar scenario where remote users ask the presenter to change the slide, scroll a bit up and down, or adjust the zoom level.
The Captured Surface Control API addresses these problems.
How do I use Captured Surface Control?
Using Captured Surface Control successfully requires a few steps, such as explicitly capturing a browser tab and gaining permission from the user before being able to scroll and zoom the captured tab.
Capture a browser tab
Start by prompting the user to choose a surface to share using getDisplayMedia(), and in the process, associate a CaptureController object with the capture session. We will be using that object to control the captured surface soon enough.
constcontroller=newCaptureController();conststream=awaitnavigator.mediaDevices.getDisplayMedia({controller});Next, produce a local preview of the captured surface in the form of a <video> element:
constpreviewTile=document.querySelector('video');previewTile.srcObject=stream;If the user chooses to share a window or a screen, that's out of scope for now—but if they chose to share a tab, we may proceed.
const[track]=stream.getVideoTracks();if(track.getSettings().displaySurface!=='browser'){// Bail out early if the user didn't pick a tab.return;}Permission prompt
The first invocation of either forwardWheel(), increaseZoomLevel(), decreaseZoomLevel() or resetZoomLevel() on a given CaptureController object produces a permission prompt. If the user grants permission, further invocations of these methods are allowed.
A user gesture is required to show a permission prompt to the user, so the app should only call the aforementioned methods if it either already has the permission, or in response to a user gesture, such as a click on a relevant button in the Web app.
Scroll
Using forwardWheel(), a capturing app can forward wheel events from a source element within the capturing app itself to the captured tab's viewport. These events are indistinguishable to the captured app from direct user interaction.
Assuming the capturing app employs a <video> element called "previewTile", the following code shows how to relay send wheel events to the captured tab:
constpreviewTile=document.querySelector('video');try{// Relay the user's action to the captured tab.awaitcontroller.forwardWheel(previewTile);}catch(error){// Inspect the error.// ...}The method forwardWheel() takes a single input which can be either of the following:
- An HTML element from which wheel events will be forwarded to the captured tab.
null, indicating that forwarding should stop.
A successful call to forwardWheel() overrides previous calls.
The promise returned by forwardWheel() can be rejected in the following cases:
- If the capture session has not yet started or has already stopped.
- If the user did not grant the relevant permission.
Zoom
Interacting with the zoom level of the captured tab is done through the following CaptureController API surfaces:
getSupportedZoomLevels()
This method returns a list of zoom levels supported by the browser for the surface type being captured. Values in this list are represented as a percentage relative to the "default zoom level", which is defined as 100%. The list is monotonically increasing and contains the value 100.
This method may only be called for supported display surface types, which at the moment means only for tabs.
controller.getSupportedZoomLevels() may be called if the following conditions hold:
controlleris associated with an active capture.- The capture is of a tab.
Otherwise, an error will be raised.
The "captured-surface-control" permission is not required for calling this method.
zoomLevel
This read-only attribute holds the current zoom level of the captured tab. It is a nullable attribute, and holds null if the captured surface type does not have a meaningful definition of zoom-level. At this time, zoom-level is only defined for tabs, and not for windows or screens.
After the capture ends, the attribute will hold the last zoom-level value.
The "captured-surface-control" permission is not required for reading this attribute.
onzoomlevelchange
This event handler facilitates listening to changes to the captured tab's zoom level. These happen either:
- When the user interacts with the browser to manually change the zoom-level of the captured tab.
- In response to the capturing app's calls to the zoom-setting methods (described below).
The "captured-surface-control" permission is not required for reading this attribute.
increaseZoomLevel(), decreaseZoomLevel() and resetZoomLevel()
These methods allow manipulation of the captured tab's zoom level.
increaseZoomLevel() and decreaseZoomLevel() change the zoom level to the next or previous zoom-level, respectively, as per the ordering returned by getSupportedZoomLevels(). resetZoomLevel() sets the value to 100.
The "captured-surface-control" permission is required for calling these methods. If the capturing app does not have this permission, the user will be prompted to grant or deny it.
These method all return a promise which is resolved if the call is successful and rejected otherwise. Possible causes for rejection include:
- Missing permission.
- Called before the capture started.
- Called after the capture ended.
- Called on a
controllerassociated with a capture of an unsupported display surface type. (That is, anything but tab-capture.) - Attempts to increase or decrease past the maximum or minimum value, respectively.
Notably, it is recommended to avoid calling decreaseZoomLevel() if controller.zoomLevel == controller.getSupportedZoomLevels().at(0), and to guard calls to increaseZoomLevel() in similar fashion with .at(-1).
The following example shows how to let the user increase the zoom level of a captured tab directly from the capturing app:
constzoomIncreaseButton=document.getElementById('zoomInButton');zoomIncreaseButton.addEventListener('click',async(event)=>{if(controller.zoomLevel>=controller.getSupportedZoomLevels().at(-1)){return;}try{awaitcontroller.increaseZoomLevel();}catch(error){// Inspect the error.// ...}});The following example shows how to react to zoom level changes of a captured tab:
controller.addEventListener('zoomlevelchange',(event)=>{constzoomLevelLabel=document.querySelector('#zoomLevelLabel');zoomLevelLabel.textContent=`${controller.zoomLevel}%`;});Feature detection
To check if Captured Surface Control APIs are supported, use:
if(!!window.CaptureController?.prototype.forwardWheel){// CaptureController forwardWheel() is supported.}It is equally possible to use any of the other Captured Surface Control API surfaces, such as increaseZoomLevel or decreaseZoomLevel, or to even check for all of them.
Browser support
Captured Surface Control is available from Chrome 136 on desktop only.
Security and privacy
The "captured-surface-control"permission policy lets you manage how your capturing app and embedded third-party iframes have access to Captured Surface Control. To understand the security tradeoffs, check out the Privacy and Security Considerations section of the Captured Surface Control explainer.
Demo
You can play with Captured Surface Control by running the demo on Glitch. Be sure to check out the source code.
Feedback
The Chrome team and the web standards community want to hear about your experiences with Captured Surface Control.
Tell us about the design
Is there something about Captured Surface Capture that doesn't work as you expected? Or are there missing methods or properties that you need to implement your idea? Have a question or comment on the security model? File a spec issue on the GitHub repo, or add your thoughts to an existing issue.
Problem with the implementation?
Did you find a bug with Chrome's implementation? Or is the implementation different from the spec? File a bug at https://new.crbug.com. Be sure to include as much detail as you can, as well as instructions for reproducing. Glitch works great for sharing reproducible bugs.
