Platform/GFX/OffMainThreadCompositing

From MozillaWiki
< Platform‎ | GFX
Jump to: navigation, search

Overview

See BenWa's blog post about OMTC (note, however that the Java part in the blog post only applies to Android).

Current status

Shipped on all platforms except Linux (almost there on Linux as of Oct 6th 2014).

Goals

The main goal is to improve responsiveness. this architecture has the following advantages over our on-the-main-thread approach:

  • Reduce main thread contention
  • Asynchronous scrolling
  • Preventing tearing
  • Asynchronous video
  • Asynchronous CSS animation

Design

Gfx layers class diagram

The content thread(s) renders web content into intermediate surfaces (layers) and sends these updates to the compositor thread through layers transactions. We call client layers the layers on the content side, and host layers the ones on the compositor side. Layer classes are mostly responsible for defining the shape of the layer tree and own Compositable objects that are responsible for the logic of how to transfer and composite the rendered web content (for instance handling tiling, double buffering, etc.). There are client and host implementations for compositables as well. For compositables to remain backend-independent, we abstract the backend specific code out behin the texture abstraction (also split between the client and host instances), which handles the lifetime of the unerlying shared texture data and provides access read/write access to it through Moz2D and TextureSource (the latter is how textures interact with the Compositor API). Layers transaction makes sure that all of the editions applied to the client layer tree during a transaction are applied "at once" in the host layer tree so that we always composite content in a coherent state. The communication mechanism is based on IPDL, which abstracts out cross-thread and cross-process communication. e10s uses the same code as regular OMTC, except that the content and compositor threads are now in separate processes.

Plan

Our highest priority platform for this project has been mobile platforms, but we want to extend this to all supported platforms: Mac, Windows (D3D9 + D3D10), Linux, and software-only (BasicLayers; possibly, there is ongoing discussion as to whether this is the best use of resources to improve the experience for 'low-end' users).

Current status

WebGL

WebGL optimizations can be made to avoid synchonization between the the cpu and the gpu. Assignee: Jeff Gilbert, Cody Brocious (Daeken)
Status: Mostly done

CSS Animations

Currently CSS animations are driven by the main thread. It'd be better if the style and layout code could assign attributes to relevant layers that would then be interpolated using specified functions on the compositor thread.

Assignee: David Baron (dbaron); previously David Zbarsky (dzbarsky)
Status: enabled on B2G, being enabled on other platforms in Firefox 41

Plugins

Plugins are already drawn asynchronously (at least on OS X), similarly to how HTML5 video works. Publishing those plugin "frames" to the compositor is directly analagous. Plugins now work with OMTC when APZ is disabled, but windowed plugin need some more work to play well with APZ (and e10s?).

Assignee: UNASSIGNED
Status: APZ+ windowed plugins need some work.

Animated images

Animated images are drawn using the main thread, but at least conceivably we could publish all the frames to the compositor and have the compositor know how to animate the images.

This could help us to animate throbbers, etc, but we don't currently have any layers support at all for animated images, so it will be more work for potentially less gain.

Assignee: Nobody Status: On hold.

Off-main-thread layers implementations

Features listed above won't work off the main thread unless we have a layers implementation that knows how to draw off the main thread.

B2G

OMTC, OMTA, APZ, tiling and async-video are enabled. GL backend only.

Assignee: Sotaro Ideka/Nicolas Silva Status: Shipped

Android

OMTC, APZ, tiling and async-video are enabled, OMTA is disabled. GL backend only.

Assignee: Benoit Girard Status: Shipped

Mac OS X

bug 756601 OMTC, tiling and async-video are enabled, OMTA and APZ are disabled (work in progress). Mostly the GL backend, and to a lesser extent basic backend.

Assignee: Matt Woodrow Status: Shipped

Linux

bug 722012 Mostly works, one reftest failure to figure out before we can enable OMTC and async-video (work in progress). Linux will use basic layers by default at least for some time, the GL backend can be enabled through a pref.

Assignee: Nicolas Silva
Status: In development.

Windows

OMTC and async-video enabled, tiling, OMTA and APZ are disabled (work in progress). Windows users get the d3d11, d3d9 or basic backend depending on the OS and driver versions.

Assignee: Bas Schouten/Nicolas Silva
Status: Shipped

Basic backend (Software-only)

This layers implementation is used when we have no hardware-accelerated layers backend; for example, when we're on a system with too-old or buggy drivers.

Assignee: Marco Castelluccio/Nicolas Silva
Status: Shipped

Tracking bug

The tracking bug is bug 598873. Tree view: [1]

Development

Development is happening on mozilla-central.

Test cases

http://people.mozilla.com/~ajuma/omtc/