Why a Canvas Tag

(By NeilDeakin)

There have been a number of requests for a canvas tag which would allow customized drawing in XUL. Much of the functionality overlaps with SVG. In fact, when I asked what people would use a canvas for, many replied that they wanted to create charts and diagrams, for which SVG is probably more suitable.

However I do think that a canvas tag is needed, if only because it seems to be one of the significant complaints against XUL from XAML or Flex supporters. I'd already created much of a canvas tag based on Joe Hewitt's earlier work. XUL:Benjamin Smedberg also did some additional work. With this implementation, the canvas allows customized drawing as well as control over layout and positioning of children. With this, it at least provides some fallback when XUL or Gecko doesn't support some widget or layout feature directly, such that developers can implement it themselves.

The Patch

Here is some information about the current canvas tag patch. Also see the Mozilla bug.

The canvas tag was intended to be used in connection with XBL to create custom tags. For example, a <rectangle> tag which draws a rectangle. Normal usage is done as follows:

<binding id="rectangle" display="xul:canvas">

Assume that Bug 119389 is fixed. The code in CSSFrameConstuctor will ensure that this will create the necessary canvas frame types. That said, one can still use the <canvas> tag directly, since CSSFrameConstuctor will fallback on the tagname in the absence of a XBL 'display' type.

Why XBL based implementations? Two reasons. First, it allows custom tags to be abstracted into a separate file. Second, the XBL <implementation> can be used to implement the two canvas-specific interfaces.

The canvas inherits from xul:box, which means it acts just like one. That means that it can have children and they will be positioned and flexed as necessary. It also handles CSS properties like borders, margins, padding, opacity, and so forth.

Canvas Drawing

There are two ways in which drawing to the canvas can be done. In buffered mode, one draws to a buffer of some specific size and the native canvas code will draw it as necessary. This is similar to Apple's canvas tag I believe, and is pretty much just like an <image> tag except that the pixels can be changed.

The actual drawing is done via a scriptable version of nsIRenderingContext which contains a variety of drawing primitives like drawLine, fillRect and so forth. It also has pixel-level drawing but tends to be too slow to draw a lot of pixels in JS, mostly due to the XPConnect bridge. Calling a setPixels function which took an array was faster, although that isn't in the current patch.

The buffered mode is suitable when the canvas isn't going to be resized, since the buffer would need to be resized in that situation.

The other drawing mode is done via a paint callback. The XBL implementation would implement nsIBoxPaintManager which contains a paint method. The paint method will be called during a paint event and will be passed a rendering context which may be drawn to. The interface nsIBoxPaintManager already exists in Mozilla today but isn't called or used for anything.

In the paint method, the code would draw whatever is desired. This technique doesn't require a buffer but the paint method will be called whenever painting needs to be done. As an extra feature, the paint method may optionally be called separately during the background phase and the foreground phase such that different drawing may be done to draw behind the children of the canvas

and in front of them. By default, only foreground drawing is done; this can be changed through the canvas box object.

The paint method is more suitable for resizable canvases since the paint method can just draw from (0,0) to (width, height). This makes it suitable for creating mock SVG-like tags such as <circle>.

Since nsIBoxPaintManager is an interface, it is also possible to implement it via a native component, if higher performance was desired. This could be used for animation and could be a rather simple form of plugin. I considered adding an attribute to <canvas> which would specify the component to be used, but the component names tend to look a bit ugly in my opinion.

Canvas Layout

The canvas also supports using a custom layout as opposed to boxes. By creating a canvas which implements nsIBoxLayoutManager one can use callbacks to position the children during the layout phase or when the window is resized. Like nsIBoxPaintManager, nsIBoxLayoutManager also already exists in Mozilla but isn't used any where.

The callback positionChild will be called for each child of the canvas. The canvas is expected to set some out parameters with the desired position and size. One disadvantage is that positionChild takes 10 arguments which might be overwhelming, but I found it necessary to pass things like the available size and canvas margins so the canvas user doesn't need to calculate these things themselves. The positionChild function will be passed the default position and size of the child according to normal xul box processing. The canvas user should change these inout arguments as desired and then return.

There were several motivations for having the layout manager part of the canvas as well, despite that XUL and CSS provide a variety of layout types already. First is Joe Hewitt's remark that the XAML GridPanel was simpler than the XUL grid tag, since XUL requires columns, column, rows and row tags as well, whereas GridPanel allows all of the cells to be placed to be direct descendants without intermediate tags.

The canvas layout manager would allow this usage by positioning the children as necessary. In fact, one of the examples I created to test the canvas tag was to implement GridPanel.

I also wanted to be able to create a timeline type of view in which items were positioned horizontally according to times associated with each item. With existing layout types, this was difficult involving a variety of stacks and scriping. I wanted something that could allow me to move the details of positioning out of the XUL and into a tag implementation in XBL.

In addition, some developers may wish to have a particular tag structure but CSS may not allow them to specify their positions as desired.

Relation to Apple's canvas tag

Apple's Dashboard has a canvas tag which can be used for drawing. There doesn't seem to be any documentation for it currently, although one can use a getContext method to get a graphics context with drawing primitives. The XUL canvas has something similar although one gets at it via canvas.boxObject.renderingBuffer.

I'm not sure if compatibility with Apple's canvas is necessary. It only works in Dashboard and not in Safari, so there won't be any canvases on the web. The only value would be if the canvas were to be formalized via the WHAT or some such group, or if there was a desire for Mozilla to be able to directly support Dashboard widgets.

One could add compatibility though by adding a method to a default canvas tag XBL binding or implementation.

Additional Notes

I wondered whether there was any security issues with the canvas. Certainly if the canvas could be used to draw over top of other frames it would be an issue. The canvas sets the clip rectangle before the paint method is called and one could be restricted from changing it in remote content if there is an issue. In this sense, the canvas wouldn't be any different from any other image. In addition, getPixel might need to be disabled or removed.

One issue is that Mozilla crashes if someone removes the canvas from the DOM during the paint or layout calls. This isn't likely to happen and is probably very difficult to fix. Anyway, the XUL tree already has this issue since it calls into the view during paint events.

Another thing I thought of was to allow one to specify the default layout type for the canvas, using stack for instance instead of box. This is relatively simple since in XUL the layout specifics are handled by a separate object hanging off of the xul frame, which could be different based on some attribute.

[Roc] Ian Hickson has some ideas about a <canvas> based on SVG, that he wants to work out in WHATWG. I don't know how this would work yet --- maybe an API to say "draw this SVG element in the canvas", with shortcut APIs to draw common elements? --- but defining and implementing it in terms of SVG would give us a powerful and "standards based" drawing API.

[Anne] That CANVAS element is defined here: Graphics: The bitmap canvas

update: That "alternative" canvas element has been implemented in a different bug, and will be enabled in Firefox 1.1 "developer preview" (whatever). Initial docs are available on devmo-test.

[Vlad] Note that while there is now a canvas as part of 1.1, the element itself is implemented for HTML only. The XUL Canvas described here would still be a great addition to XUL -- it can share the back-end rendering context code with the HTML Canvas, while also adding support for child positioning and the other XUL-specific elements.