NPAPI:AsyncDrawing
Contents
Status
Accepted, ready for implementation.
Contributors
- Last modified: April 19, 2012
- Authors: Bas Schouten (Mozilla Corporation), Josh Aas (Mozilla Corporation)
- Contributors: Robert O'Callahan (Mozilla Corporation)
Overview
This specification allows plugins to draw asynchronously to surfaces that are potentially located in video memory.
Negotiating Async Drawing
For documentation on negotiating drawing models, see NPAPI:Models. This specification covers all async models, which use the same drawing mechanics with platform-specific data structures. The following models are currently specified:
- NPDrawingModelAsyncBitmapSurface (NPDrawingModel = 7)
- NPDrawingModelAsyncWindowsDXGISurface (NPDrawingModel = 8)
Support query variables are:
- NPNVsupportsAsyncBitmapSurfaceBool (NPNVariable = 2007)
- NPNVsupportsAsyncWindowsDXGISurfaceBool (NPNVariable = 2008)
- NPNVpreferredDXGIAdapter (NPNVariable = 2009)
Since these are the first new models for Windows and Linux the existing models will be assigned names:
- NPDrawingModelSyncWin (NPDrawingModel = 5)
- NPDrawingModelSyncX (NPDrawingModel = 6)
General Async Drawing Mechanics
Any async drawing model will allow plugins to create one or more surfaces for drawing. The plugin will tell the host which surface is current at any given time and the host will display the current plugin surface for as long as it is current. While a surface is current the plugin is not allowed to modify it. This system allows the plugin to choose whether to double or triple buffer.
Surfaces are specified via the NPAsyncSurface structure. This structure contains a pointer to a platform-specific resource of the type defined by the drawing model:
/* These formats describe the format in the memory byte-order. This means if
* a 32-bit value of a pixel is viewed on a little-endian system the layout will
* be 0xAARRGGBB. The Alpha channel will be stored in the most significant
* bits. */
typedef enum {
/* 32-bit per pixel 8-bit per channel - premultiplied alpha */
NPImageFormatBGRA32 = 0x1,
/* 32-bit per pixel 8-bit per channel - 1 unused channel */
NPImageFormatBGRX32 = 0x2
} NPImageFormat;
/* This is version 0 of the structure. */
typedef struct _NPAsyncSurface
{
uint32_t version;
NPSize size;
NPImageFormat format;
union {
struct {
unsigned char *data;
uint32_t stride;
} bitmap;
#ifdef XP_WIN
HANDLE sharedHandle
#endif
};
} NPAsyncSurface;
New surfaces can be initialized via a new function called NPN_InitAsyncSurface:
NPError NPN_InitAsyncSurface(NPP instance, NPSize* size,
int32_t format, void* initData,
NPSurface* surface);
When a plugin is done with a surface the surface should be finalized with NPN_FinalizeAsyncSurface:
NPError NPN_FinalizeAsyncSurface(NPP instance, NPAsyncSurface* surface);
What exactly this means depends on the drawing model's ownership policy and the implementation. If the finalized surface is current, this function will return an error.
The currently displayed surface can be set via a new function called NPN_SetCurrentAsyncSurface:
void NPN_SetCurrentAsyncSurface(NPP instance, NPAsyncSurface* surface, NPRect *changed);
Surfaces cannot be modified while they are current. A modified rectangle may be provided in the changed argument to give the host a hint on which part of the surface was modified with respect to the previous current surface so that the browser can choose to optimize the composition process. There is no guarantee only this rectangle will be recomposed though and the entire surface needs to be valid.
SetCurrent can take NULL as pthe surface. This removes the previous current surface. The plugin will render nothing. This should be used before finalizing the last surface. All three functions must be called on the plugin thread.
Reentrant calls from the browser into the plugin do not occur during a SetCurrent invocation from the main thread. The browser is expected to implement SetCurrent using a locking protocol that cannot block for long. For example, SetCurrent might block only while the previous surface is being read by the browser's compositing.
Throttling Painting
To enable the plugin to match its frame rate to the browser's, the browser notifies the plugin each time it has composited a frame that used the plugin's current surface. This notification fires even if the plugin's current surface is NULL.
void NPP_DidComposite(NPP instance);
Hidden Plugins
While a plugin is in a hidden tab (or is hidden for some other reason), the browser doesn't have to fire NPP_DidComposite. When the plugin becomes visible, the browser won't wait for the plugin to re-render before compositing --- that would defeat the purpose of an asynchronous API. This could mean displaying an old, stale frame of the plugin for a moment. A plugin can mitigate this by calling SetCurrentAsyncSurface at a low rate even while DidComposite is not being received. In some situations it might be appropriate for a plugin to set the current surface to NULL while it's hidden (NPP_SetWindow having set an empty clip-rect); this would cause the plugin to render nothing instead of a stale frame when it becomes visible.
NPDrawingModelAsyncBitmapSurface
This model should be available on all platforms.
Plugins should create a bitmap surface by calling NPN_InitAsyncSurface with a NULL value for initData. If an NPImageFormat is not supported then surface creation will fail.
Plugins should destroy bitmap surfaces by calling NPN_FinalizeAsyncSurface. This will allow the browser to release the bitmap memory.
This drawing model will only be valid on Windows Vista and higher in order to simplify hardware accelerated surface sharing.
Plugins should create a Windows shared surface by calling NPN_InitAsyncSurface with a NULL value for initData. The resulting NPAsyncSurface will point to a HANDLE that can be used, for example, through OpenSharedResource in order to create a texture for the user. In order to allow fast drawing to any hardware surfaces, the host will acquire the handle from IDXGISurface::GetSharedHandle. This shared handle will represent a texture which is usable as a render target and is valid as a shader resource. The plugin can open this shared handle as a texture and then use it as a render target for any drawing operations. In order to synchronize GPU access to the surface the resource created will support the IDXGIKeyedMutex interface, the key used for both acquiring and releasing sync is 0.
Plugins should finalize Windows shared surfaces by calling NPN_FinalizeAsyncSurface when they are done with the surface.
Creating the Device
On dual-GPU systems, the browser and plugin process may unintentionally bind to different GPUs if default DXGI parameters are used. When this happens the plugin's draw calls will most likely fail, or crash. To address this, it is necessary to open the browser's shared DXGI surfaces on the same adapter. The adapter can be found using NPNVpreferredDXGIAdapter. For example,
static IDXGIAdapter1*
FindDXGIAdapter(NPP npp, IDXGIFactory1* factory)
{
DXGI_ADAPTER_DESC preferred;
if (NPN_GetValue(npp, NPNVpreferredDXGIAdapter, &preferred) != NPERR_NO_ERROR) {
return NULL;
}
for (UINT index = 0; ; index++) {
IDXGIAdapter* adapter = NULL;
if (FAILED(factory1->EnumAdapters1(index, &adapter)) || !adapter) {
return nullptr;
}
DXGI_ADAPTER_DESC desc;
if (SUCCEEDED(adapter->GetDesc(&desc)) &&
desc.AdapterLuid.LowPart == preferred.AdapterLuid.LowPart &&
desc.AdapterLuid.HighPart == preferred.AdapterLuid.HighPart)
{
return adapter;
}
adapter->Release();
}
return NULL;
}
The adapter returned by this function can then be used with any DXGI device creation function, such as D3D10CreateDevice. Failure to negotiate the adapter properly (or failure to use an IDXGIAdapter1) will result in undefined behavior.
If no matching adapter can be found, the safest course of action for the plugin is to switch to a different drawing model (such the AsyncBitmap model).
Sample Code
This sample code illustrates usage of this drawing model:
ID3D10Device *pDevice10;
NPAsyncSurface *npFrontBuffer = new NPAsyncSurface;
NPAsyncSurface *npBackBuffer = new NPAsyncSurface;
ID3D10Texture2D *frontBuffer;
ID3D10Texture2D *backBuffer;
void Init() {
... Initialize pDevice10 ...
NPSize size;
size.width = pluginwidth;
size.height = pluginheight;
NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, NULL, npFrontBuffer);
NPN_InitAsyncSurface(instance, size, NPImageFormatBGRX32, NULL, npBackBuffer);
pDevice10->OpenSharedResource(npFrontBuffer->handle,
__uuidof(ID3D10Texture2D),
(void**)(&frontBuffer));
pDevice10->OpenSharedResource(npBackBuffer->handle,
__uuidof(ID3D10Texture2D),
(void**)(&backBuffer));
DrawFrame();
}
void DrawFrame() {
IDXGIKeyedMutex *mutex;
backBuffer->QueryInterface(&mutex);
mutex->AcquireSync(0);
... Draw to backBuffer texture ...
mutex->ReleaseSync(0);
mutex->Release();
NPN_SetCurrentAsyncSurface(instance, npBackBuffer);
ID3D10Texture2D *tmp = frontBuffer;
NPAsyncSurface *npTmp = npFrontBuffer;
frontBuffer = backBuffer;
npFrontBuffer = npBackBuffer;
backBuffer = tmp;
npBackBuffer = npTmp;
}
void NPP_DidComposite() {
DrawFrame();
}
void Shutdown() {
frontBuffer->Release();
backBuffer->Release();
NPN_SetCurrentAsyncSurface(instance, NULL);
NPN_FinalizeAsyncSurface(instance, npFrontBuffer);
NPN_FinalizeAsyncSurface(instance, npBackBuffer);
delete npFrontBuffer;
delete npBackBuffer;
}
