Changes

= xul templates = This is the place were we want to collect ideas about how xul templates could evolve in the future. To retain history of this discussion, I'm leaving this alive and open [[XUL:Template Builder Arch]] for the new stuff. === Comments [[XUL:Axel Hecht]] === I myself tried to learn templates, and I wouldn't say that I succeeded. I got one or two templates working, that's about it. So one major wish I had would be to lower the complexity. To me, the complexity of templates comes from the many flavours of it's current syntax. It's like the perl of xul. There are more ways to your goal than persons to implement it. Sadly enough, you rarely hit a working syntax by chance :-). So I have three main targets, one is to cut down the available syntaxes to, say, two or three. And then there would be extensions to templates to cope with the [http://www.axel-hecht.de/mozwiki/RDF new developments in RDF land] like typed literals and literals with xml:lang. And then there are extensions that enable templates to work on more usecases. As far as syntax goes, I would like to move the template attributes out of xul elements and put them on the xul:template element. That element is rather useless right now from a markup point of view, and having the datasources and ref attrs on xul elements is an unnecessary restriction of templates. Removing the template markup from xul elements enables us to have non-xul elements as template container as well, at least from the markup point of view. I have no clue how much the template builder relies on XUL content/document/contentsink. Evaluating that will be essential for mixed content documents. (And generating xul thru xslt, yeah, I know.) About RDF conformance, RDF should support typed literals as well as string literals with xml:lang in the not-so-distant future. That should be reflected in the template architecture. That is, rules should be able to specify whether they want to test for the date "may 5th 2003" or the string. Or the int 5 or the string "5". I have no idea if the current template syntax supports testing for arc targets that are resources and not literals. What should happen with xml literals as target, would we want to enable generating content from that directly? One usecase I have when it comes down to templates is the comparison of test results. Which is rather painfull, as that rdf we use right now has to have different predicates for the first and the second result. It would be nice if we could use distinct datasources for rules. So one could say if the first datasource has a <urn:test-result> arc to "true" from <urn:tests:cat1:test5> and the second datasource has a <urn:test-result> arc to "false", the first column would pop up green, the second one red, if we'd specify the rules accordingly. Right now, we mangle the <urn:test-result> arc while saving to be <urn:orig-test-result> and use that to test for the second column. That works fine as long as one just compares an unsaved test with one from disk, but we cannot compare two result sets from disk. I'd like to have named datasources for that, as can be seen in the RelaxNG schema below. I opted for compact notation, as it is really more compact. But I have both an xml notation on my disk, too. What we should use in the end (if at all) should depend on the tools available. <pre> # A RELAX NG compact syntax pattern # for an XUL templates. namespace xul = inherit XulTemplate = element xul:template { # not sure if we want interleave or not for the child elements # datasources attribute or elements # not sure how to specify, that we want at least one, # but allow both attr and element attribute datasources {}?, element xul:datasources { attribute uris {}, # not sure if the token type is right here attribute name { token }?, } # a set of rules element xul:rule {}+ # to be done } datasourcesList = list {token} </pre> === Comments [[XUL:Vlad Vukicevic]] === Some details of my fight with the current template builder and cases where things fail are in [http://bugzilla.mozilla.org/show_bug.cgi?id=248015 bug 248015]. Looking through mozilla/**/*.xul for current usage of templates, the vast majority are are simple one-rule templates just to autogenerate content. Multi-rule templates mostly deal with containers -- special casing empty and non-empty containers from the rest of the elements. Most templates also use the "simple" syntax as opposed to the extended syntax (conditions, bindings, etc.). The few cases where the extended syntax is used, e.g. [http://lxr.mozilla.org/aviarybranch/source/mail/base/content/messenger.xul mozilla/mail/base/content/messenger.xul], they can (I think) be rewritten using the simple syntax, but use the extended syntax to simplify the actions section (so that "?foo" can be used instead of "rdf:http://some/name/space/Foo"). (Note: for reference, the current "simple syntax" is the same as taking the simple conditions and making them complex ones, prepending a <content uri="?root"> <member container="?root" child="?val"> and then hanging the conditions and bindings off ?val.) Summary: 1. Most templates are used to generate simple content 2. More complex templates generally only include container state (iscontainer/isempty) and specific arc tests ("rdf:type" == "myns#Foo") 3. Extended syntax rules generally are only used so that aliases can be created for variable bindings, so that something like: properties="specialFolder-?specialFolder biffState-?biffState isServer-?isServer newMessages-?newMessages hasUnreadMessages-?hasUnreadMessages subfoldersHaveUnreadMessages-?subfoldersHaveUnreadMessages isSecure-?isSecure serverType-?serverType redirectorType-?redirectorType noSelect-?noSelect imapShared-?imapShared" doesn't become worse. The remaining extended examples do things like: <conditions> <content uri="?uri"/> <triple subject="?uri" predicate="http://home.netscape.com/NC-rdf#subheadings" object="?subheadings" /> <member container="?subheadings" child="?subheading"/> </conditions> and then have bindings based on arcs from ?subheading, or things like: <conditions> <content uri="?uri" /> <member container="?uri" child="?subheading" /> <triple subject="?subheading" predicate="http://home.netscape.com/NC-rdf#purp" object="?purp" /> </conditions> where the sole purpose of the triple and of ?purp is to ensure that an outbound #purp arc exists (only ?subheading is used in bindings). WRT the template element, I'm not sure that moving the datasources and ref attributes to the template element from the base xul element is a good idea -- being able to reuse the same template but with different data sources and possibly different refs seems like a useful thing. Taking bookmarks as an example, given a hypothetical RSS feed data source, the bookmarks sidebar could aggregate the RSS data source whereas the bookmarks manager tree would not aggregate it so that the feed bookmarks can be manipulated without them looking like containers. For the ref attribute, keeping it on the element would give an easy way to create subviews of generated content, though I'm less convinced that this is useful compared to datasources. === Comments [[XUL:Benjamin Smedberg]] === Can we step back just a little bit farther? I believe that whatever data-binding mechanism we use, <strong>should not be reflected in the source DOM</strong>. This is analogous to the way that XBL works: leave the source DOM intact, and have the abstraction of a "presentation DOM". We wouldn't need the weird dichotomy between tree templates and content templates. This may sound radical at first glance, but we already have a lot of the technology in place for XBL already. Secondly, making all our data corform to RDF is also silly. Templates, by their nature, create a hierarchical structure, and RDF is a graph structure, not hierarchical. I think that there can be an intermediate "hierarchy interface" that could be implemented by RDF, XML, relational database, customized mail backends, etc. DoronRosenberg was doing some work on this. My question would be, at that point, whether we want a single query language for RDF/XML/SQL/custom whatever, or whether we want to use separate query languages for each data type. Prolog comes to mind for a unified query language. Many data-specific query languages are available: use XPath for XML "datasources" (is xpath rich enough, can it do variable binding?), and use one of the many RDF query languages for RDF, use SQL for relational data, etc. === Comments [[XUL:Axel Hecht]] === Re [[XUL:Vlad Vukicevic]], are you talking about overlays here? Or xbl? Not sure what the bookmarks actually use. Anyhow, I'm not sure if one couldn't just overlay the template element instead of it's parent as well. Re [[XUL:Benjamin Smedberg]], I'd say that adding three query languages and source data formats is not really helping in making templates easier. Just like prolog doesn't seem to be something that too many webdevelopers are familiar with. Would you allow mixing XML, RDF or SQL datasources? Would you allow mixing the variable declarations? Would we run into datatype conflicts? Re [[XUL:Benjamin Smedberg]], wouldn't removing the templated content from the DOM tree make using CSS for styling harder? Or even impossible? I'm not sure how anonymous content does that right now, but it sounds like this could be hard. (Which doesn't matter that much for trees, as they only allow very limited styling in the first place.) === Comments [[XUL:Vlad Vukicevic]] === Re [[XUL:Axel Hecht]]: I was referring to overlays, though I'd assume the same thing would apply to xbl. I am, however, refering to non-inline template usage, i.e. <hbox template="some-template" /> <template id="some-template">...</template>, vs. <hbox><template>...</template></hbox>. In the second case, I can see where moving the ds/ref tags onto the template element would make sense, but I'd suggest that that syntax be removed in favour of the first syntax (to encourage template sharing, mainly, and also to keep the "content" area in a xul document free of template-related markup). You mentioned originally that you'd like to cut down on the many flavours of template syntax; which ones are you thinking to keep, and which to remove? (I see simple and extended, in inline and non-inline forms; I'd get rid of the inline forms.) [[XUL:Benjamin Smedberg]]: I agree that it would be nice to not have all data end up passed around as an RDF graph; however, does creating another data representation layer just for templates makes sense, as most data sources within the framework already know how to convert themselves to RDF? I would like to see more explicit control in the template builder on what happens for containers; right now, the same template gets applied recursively to all children (in a member binding) without being explicitly specified in the template. Making this be an explicit part of a template spec could help clarify what the template is actually doing. It would also let you apply a different template to children than you would apply to the first-level elements, which can lead to better template sharing. === Comments Scoubidou === I would like to access to a SQLite database with PHP or XUL. It would be interesting to make embedded applications on CD-ROM. === Comments [[XUL:Axel Hecht]] === From the point of learning curve, we cannot do without the inline template syntax. I personally would like to get rid of the "simple" form of it. From a learning-xul point of view, I hate the non-inline syntax. And how does that work with modifying the DOM of a template (which I guess is not working right now, but it should work). Of course, one could still have a <pre> &lt;template use="aTemplateID">&lt;/template> </pre> if I'd raise my initial proposal again. IMHO, it is really unfortunate that XUL elements and templates are so intermingled. === Comments [[XUL:Benjamin Smedberg]] === [[XUL:Axel Hecht]]: Removing templated content should not affect the CSS styling rules; they would behave exactly as they currently do for XBL (CSS is applied to the "presentation DOM", not the source DOM). [[XUL:Vlad Vukicevic]]: I agree that we need much more explicit control over containers and recursion. I think that recursion should always be explicitly specified, if that behavior is desired (using a self-referencing template). [[XUL:Axel Hecht]]: my goal was to increase developer familiarity by using common query languages (especially SQL). I'm conflicted, but I thought I would throw the idea out... doron should comment more. Imagine a SQL template: &lt;template parameters="name"> &lt;query type="application/sql" > SELECT bar, baz FROM table WHERE foo='%name' &lt;/query> &lt;rule condition="bar=1"> &lt;!--insert conditional template content here, for rows where bar=1--> &lt;/rule> &lt;rule> &lt;!--if the row didn't match any of the conditions above, insert this content--> &lt;/rule> &lt;/template> You could use an RDF-specific query language: &lt;template parameters="name"> &lt;query type="application/rdql+bsmedberg"> SELECT ?x, ?y WHERE (?x, &lt;http://somewhere/pred1&gt;, ?name) WITH (?x, &lt;http://somewhere/pred2&gt;, ?y) &lt;!-- this is an optional arc... ?y will be null if there is no matching arc this is not part of RDQL, but we need something like it --> &lt;/query> &lt;rule condition="y"> &lt;!--insert conditional content here, if y is non-null--> &lt;/rule> &lt;rule condition="x='&lt;someuri&gt;'"> &lt;!--insert conditional content here--> &lt;/rule> &lt;!-- we don't have any default rule, this time --> &lt;/template> === Comments GervMarkham === Axel asked me to add a few thoughts. It would be nice if it were possible to add/remove template rules using DOM manipulation. Currently, if you try this (e.g. removing a <triple>) Mozilla crashes. Presumably there's an assumption somewhere that things will stay the same. This isn't useful enough to make it worth a lot of work, but if it falls out of the new work, cool. I'd also like a way to better specify when <content> is triggered in terms of <triple>s. Full Boolean logic would be nice. Currently, you can have AND by having multiple triples, and OR by having multiple rules (but then you need to duplicate your content, which is a right pain if it's complex.) You can also sort of have NOT by having an empty rule. But there's no way to do proper combinations. [One could argue that the right thing to do instead is to manipulate the underlying datasource to add arcs representing each of the conditions - i.e. move the calculations out of the template system.] Lastly, the way recursion works is currently completely mindblowing. I use it myself, and I _still_ don't understand it. === Comments [[XUL:Vlad Vukicevic]] === For the inline syntax, how about supporting only as [[XUL:Axel Hecht]] suggests, maybe modified to something like: <pre> &lt;somexulelement> &lt;template use="myTemplate" datasources="rdf:blah" ref="some#thing" /> &lt;/somexulelement> ... &lt;template id="myTemplate"> ... template body ... &lt;/template> </pre> I wouldn't support a full template declaration under somexulelement, just a reference with datasources etc. (which would no longer live as attributes on the xul element, as you suggested originally). I don't think this would lead to a steeper learning curve; it would be simpler than the current situation of having two different ways to specify what template should be applied where. === Comments GuruJ === I would be loath to make any changes that break the existing template system. Vlad talks about non-inline templates; why not develop an system that works along the same lines as overlays, eg. instead of <pre> &lt;?xul-overlay href="chrome://app/content/appoverlay.xul"?> </pre> use <pre> &lt;?xul-template href="chrome://app/content/apptemplate.xul"?>? </pre> This wouldn't be any harder to grasp than the current overlay system. Additionally, by abstracting templates into their own file, template syntax can be extended or changed without breaking existing programs. Maybe call them DDOs (data-driven overlays) or something if you want to make people think of them differently. Benjamin: I think Mozilla should be able to use XML files as a native RDF datasource, but IMHO I don't think that SQL is appropriate for an XML-based system. If we had a canonical data transformation that would allow people to use a standard RDF template for simple XML files, I think this could lower the barrier to entry, eg. <pre> &lt;root> &lt;item setting="1" /> &lt;/root> </pre> could become: <pre> &lt;rdf:Seq id="root"> &lt;rdf:li> &lt;rdf:Description xml:elem="item"> &lt;xml:properties> &lt;rdf:Seq> &lt;rdf:li> &lt;rdf:Description xml:property="setting" xml:value="1" /> &lt;/rdf:li> &lt;/rdf:Seq> &lt;/xml:properties> &lt;/rdf:Description> &lt;rdf:li> &lt;/rdf:Seq> </pre> === Comments [[XUL:Axel Hecht]] === I don't think that PIs are the right way to go. Their DOM is really poor, which makes it harder to do the stuff like Gerv wants (editing the DOM) (which should of course work instead of crash). I'm not sure either that PIs are the right concept, as you can't specify their grammar, at least not in DTDs. That will make (pseudo-)visual editing a mess. We want to have the chance to do single file XULlets (this name is dancing, at least to me, hey, souds like motown), which would rule out the separate file approach. Talking about visual editing, templates should provide a mean to disable them via the DOM, IMHO. That way you can create a visual editor without having all kinds of messy layouts while you edit the template. Edit, test, make up your mind why it does what it does, switch back off and edit again. This should work on a per-instant basis, you don't want your generated content to disappear just because you edit some on another part of your UI. Benjamin, I'm sure you can answer this with a blunt "no", but does moving the generated content into presentation DOM have any impact on templates inside templates? IIRC, that works right now, and it should work IMHO. === Comments [[XUL:Vlad Vukicevic]] === (Axel, are you sure templates inside templates work? I just tried to get it working and failed -- content for the sub-templates seemed to be on the verge of getting generated, but it bailed with some assertion or another.) I'm wondering, looking back over all this... people already know how to get at their data from JS. Why not take advantage of that? Implement a way to call a JS function which will return: null (don't generate content); an object whose properties are used for replacement within the generated content; or an array of objects, indicating to repeat the content for each element in the array, using the object's properties as replacement values as previously. The mapping content could be recursive (via id/refs). I'm not sure how you'd handle dynamic content with a scheme like this; the template builder now keeps track of what chunks of content were generated for what resource, so that it can remove/regenerate, etc. I think something like that could be implemented on top of the above, but I'm not positive; I haven't thought the idea out fully. It also might be slow. The advantages would be pretty huge though, including far easier debugging (just use dump(), Venkman, whatever), decoupling from RDF, the potential for complex rules... with a good set of helper functions, we could probably get the simple usage cases down to a very concise and natural syntax. === Comments [[XUL:Jan Varga]] === I strongly support the idea of having XML, RDF and SQL as data sources. RDF is just not suitable for everything. I could also help with the implementation, but we need a decent spec first. === Comments CurtisBuys === Forgive me as I'm new to XUL Development and XUL Templates, but I think it would be wonderful if it were possible to use XSLT inline with XUL for 'templated' XUL controls. With this 'source' file for the template: <pre> <mypage> <section title="hello!" href="/hello.html" /> <section title="goodbye!" href="/goodbye.html" /> </mypage> </pre> It would be nice if I could code this in the XUL file: <pre> <xul:vbox> <xsl:template xul:src="http://localhost/test.xml" xul:type="text/xml" match="/mypage/section"> <xul:button label="{@title}" oncommand="alert('Loading {@href}');/* ... */" /> </xsl:template> </xul:vbox> </pre> (where <code>xul:src</code> and <code>xul:type</code> are some kind of new attributes directing the rendering engine to the the source XML file) And have the resulting display be rendered as though the original XUL had been: <pre> <xul:vbox> <xul:button label="hello!" oncommand="alert('Loading /hello.mfsite');/* ... */" /> <xul:button label="goodbye!" oncommand="alert('Loading /goodbye.mfsite');/* ... */" /> </xul:vbox> </pre> Perhaps this type of transformation is already possible and I just don't know about it. I did find this [http://xulplanet.com/forum/viewtopic.php?t=916 XULPlanet forum post] but the idea was to generate a new XUL file from an existing XML file. I would prefer to be able to perform a number of templated inline transforms from multiple source XML files to result in the final XUL being rendered. Perhaps keeping this template idea in the XSLT namespace could allow inline XSLT templates to exist side by side with existing XUL RDF templates and whatever future template offering XUL will have to offer. === Comments NeilDeakin === It isn't clear whether templates and databinding would refer to the same thing or not. Templates traditionally refer to the generation of a repeated block of content, one block per available result. Databinding refers to associating data to an element or an attribute or property of an element. In my opinion, the two features are too similar, that, in my mind, they both are part of the same thing. So I would see both being implemented using the same set of features. Here is a list of the kinds of things a databinding/template system would need to be able to do: #Support different kinds of datasources, as listed below. #Be able to select any part of the data, for example a specific set of attributes. #Expressions which can return substrings, perform relational operations, etc... For instance, given a set of records, display the first four characters of a property in a label. #Display different content based on the record. For instance, different style for negative numbers, or a separator for bookmark separators. #Support generating different content inside the element from the same datasource. For instance, a grid of results, where a grid cell contains a menulist with items from the datasource dependant on the cell data. #Support aggregation of datasources. #Recursion, such that nested nodes can be generated using the same template. #AutoUpdating - when the data is modified the template results may optionally be regenerated automatically. This needs to be smart enough to only regenerate the parts of the data that would need to change. There are a number of different kinds of datasources to support: #RDF - graph structure, could be of any form. #XML - tree structure, hierarchical nodes each with any number of fields. #Relational database - table structure, a set of results all with a set of predefined fields. #Web Services - some mapping to web service calls. #Javascript Objects - retrieve data from JS objects such as arrays or objects. May want to require a interface be implemented such that native objects may be used as well. In general, I think it boils down to three different structures, tables, trees and graphs, where XML would be used for trees and RDF for graphs. Other types of datasources would implement an interface for one of the three dataource types. On possibility is to only use one type of data structure and create wrappers for the others, but, in my opinion, doing so would make one of them easier, faster and better than the others. There are three common ways of doing databinding/templates, described below. I think it would be useful to create examples of using different template systems, for instance to build an RSS reader, or the Amazon Browser, and compare to see what works best. '''Query based''' This type is probably the easiest for a developer to understand. A query, SQL for instance, is placed in the document along with info about the datasource. When the statement is executed, content is generated from a template, one block repeated for each available record. Specific attributes bind tags to fields in the returned results. A very simple form of this can be found in IE databinding. However, it is very limited; it can only generate repeated results in an HTML table or select tag. With this model, the SQL statement is executed and results are returned. The content builder then generates content from the results. This is essentially a two step process: generate results, then generate content. If more information needs to be retrieved from the datasource, another statement and query must be done. This means that the query must be sufficient enough to return all needed data. This is a disadvantage of this approach. For instance, if you wanted to display further details based on one returned item which the user has selected, another query must be done. This model is fairly easy for developers to understand since many are already familiar with SQL, and if not, could easily understand the basics since SQL tends to be pretty simple and descriptive. There isn't really an SQL language for XML however. There's XQuery but that isn't similar to SQL and is pretty much a programming language in itself. One could use XPath instead, although I'd think that something similar for all datasources would be better. '''Selector rule based''' This type involves the use of a set of selector rules along with matching content to generate. The builder is either given a single starting point, or a list of values, and applies the selector rules to the values. When a rule matches, the content associated with that rule is generated. Rules that don't match do not need to be evaluated. This approach allows the data to be retrieved only when necessary. This is perhaps awkward when dealing with a relational database backed datasource, however, it is more natural for XML and RDF datasources. For XML, the rules can be a set of XPath expressions that the navigate the source document when needed. Recursion is simple with this model since the rules can be just reapplied recursively. This approach is used by the existing XUL templates and well as XSLT. This model may be more difficult for a developer to understand than a simple query approach. An advantage is that is allows very different content to be built depending on the data. The nature of this approach is such that it is expected that different content is desired. As such, a simple case where only one rule is needed can be more complex than necessary. For instance, without the simplified XUL template syntax, a simple rule involves much extra content. In addition, if the content generated by two different rules only differs slightly, the involves more code than necessary. The advantage though is that only those expressions that need to be evaluated are executed. In the query based model, the entire query must be performed. Also, for dynamic updating, only a portion of the expressions that would be affected by a change to the datasource need to be examined. XUL templates have a set of variable bindings that are assigned when a rule matches, and those variables may be used in the content to get values from the datasource. However, it doesn't allow more complex values that aren't directly associated with the current result being evaluated. XSLT is generally only applicable to XML. For other data structures, something else would need to be used. '''Attribute binding based''' This model involves the use of attributes that contain expressions which retrieve data and set them on those attributes. For instance, an expression which calculates an age and assigns it to a label. The tag with the attributes has a data context which specifies a reference point for use in calculating the results of the expressions. Additional tags may be used to filter content or specify where repeating blocks are used. The syntax may also allow other parts of the document beside the attributes to be bound to data. This is the model used by XAML, Flex and the template language I created called ReoPath. In this model, the tags in the document are evaluated for expressions. If an expression is used, is is evaluated and the result set in the appropriate place. Processing then continues to the tag's children. In the case of XAML, XPath or a similar langauge is used and the data context must be set explicitly on the tag or one of its ancestors. For Flex, script may be used which may bind to any type of data. As such, Flex does not require the use of datasources in order to use data binding, although there are objects which may be used like datasources. With Reopath, the context is set from the result of a parent expression and is suitable when generating child content that is dependant on earlier content. XAML and Flex only allow repeating content to be used with certain kinds of elements, in general, listboxes, menus and grids. With other elements, only a single value is associated. XUL Templates and Reopath, on the other hand, do not treat certain elements specially, and allow any content to be used in a template. XAML's visual tree feature, however, alleviates this issue somewhat by allowing arbitrary content to assigned as the content of the list or menu's items.