Firefox/Projects/Firefox Start/Snippet Service

From MozillaWiki
Jump to: navigation, search

The about:home snippet service is a simple, highly-cached content management service. It is intended to assemble and deliver content snippets to the about:home page in Firefox.

The content delivered is determined by details about the installation of Firefox requesting content - including mainly details about the browser's build, locale, platform, and distribution channel, but not the person using the browser.

Overview

Scalability vs content agility

A special note here: Although we can create and edit snippets in the service at any time, it may take up to 48 hours for changes in production to be reflected in browsers. This is because:

  • Firefox only checks every 24 hours;
  • and Zeus caches content from the service for 24 hours.

So, in the worst case, it's possible for those two time periods to stack up to a 2-day delay in content delivery.

The reason for this is that this service can be potentially hit by every Firefox user, so both the server and client sides are built to minimize the pressure on Mozilla resources.

About about:home

about:home is a page built into Firefox 4 and above. By default, it fetches content from the production snippets service once every 24 hours. If content is available, it gets is stashed in localstorage for the page and displayed from there until the next fetch.

If no content is available, the browser has a built-in default set of snippets for display.

See also:

Source code for service

If you'd like to contribute or just run an instance, get the source from here:

See bug 592431 for historical context on this service.

Testing Snippets with Local or Stage Service

For testing against staging (ie. https://snippets.allizom.org) or a private development instance of the server (ie. http://localhost:8000), you'll need to tell Firefox to use these sources instead of the regular snippet service. There are two ways to accomplish this: manually and using the snippets switcher plugin.

Manually Updating Snippet Source

  1. Navigate to about:config
  2. Search for browser.aboutHomeSnippets.updateUrl
  3. Change the value accordingly. For the stage server, change the url to https://snippets.allizom.org.
  4. If you only want to retrieve a specific snippet, replace %NAME% (including the %s!) with the appropriate value.
  5. After saving the value, restart the browser.
  6. Navigate to about:home.
  7. Open the Web Developer > Web Console (ctrl+shift+k).
  8. If testing on Firefox <= 21.0, enter localStorage['snippets-last-update'] = 0 or if testing on Firefox >= 22.0 enter gSnippetsMap.clear();
  9. Refresh about:home. You should now see the new snippet.

Snippets Switcher Plugin

The Snippet Switcher is currently broken, do not use it! If you do, it may wipe out ```browser.aboutHomeSnippets.updateUrl``` and you'll have to reset it using the Recovery url below.

The snippets switcher plugin code is available here:

The xpi file is available here:

The plugin works by modifying the aboutHomeSnippets.updateUrl preference and updating localstorage variables accordingly. (See bug 603674 for more info).

Recovery

If you've lost the original value of ```browser.aboutHomeSnippets.updateUrl```, reset it to: 

https://snippets.mozilla.com/%STARTPAGE_VERSION%/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/

Then, follow the Manual Update instructions above to preview your snippet.

Getting access to the production snippet service

This resource is highly protected, since it serves up content to all installations of Firefox. Access is limited to Mozilla employees, so file a bug or ask someone who knows about the snippet service (eg. cmore@mozilla.com).

Managing snippet service content

Snippets

A snippet is an arbitrary block of content - which includes any combination of HTML, CSS, and JavaScript. Snippets associated with client match rules matching the current request URL are included in the response.

All HTML content must also be well-formed XML.

Since multiple snippets can be included in the same response, the Sort order priority field is used to provide a sort index. So, for example, if you have some JavaScript to include after all snippets, a value of 9999 will place it after the default of 0. If you have CSS to place before any other snippets, a value of -9999 will place it before the default of 0.

Considerations for external media used in snippets

Any HTML, CSS, or JavaScript can be used in snippets. However, any references to external media, such as images, should be encoded as data: URIs so that the snippets can be self-contained.

For example, a tool like The data: URI kitchen can be used to convert an image file into a data: URI.

If an external media resource is necessary, such as a video or an image too large for use as a data: URI, it will need to be hosted somewhere that can handle the potential traffic of every Firefox user requesting the URL. This means using a CDN or other high-capacity web host.

Localization

At present, localized snippets have no special handling in the service. They are just snippets with locale-specific client match rules. More work needs to be done here for bulk handling and grouping with an eye for l10n.

There are some undocumented tools available to help generate localized snippets and associated rules from an import manifest and translations attached to a bug.

The easiest way to do this right now is to:

  • run a local copy of the snippet service;
  • solicit locallizers to provide translations in text files as attachments on a bug
    • see bug 636815, ensure locale included in parens in title - eg. (en-US)
  • create an import manifest (no docs or spec on this yet);
  • run `manage.py importfrombug` against the import manifest to generate snippets and rules in a local database;
  • verify snippets were created as expected, export snippets (and rules) as JSON, upload the JSON to staging

Here are some raw pointers to code and examples of a recent import job:

Client match rules

In order to get snippets served to appropriate locales and builds of Firefox, snippets are served using client match rules. These are rules matched against the URL of a request to the service, which should contain the following client details:

  • Product name (eg. Firefox)
  • Product version (eg. 4.0b12pre)
  • App build id (eg. 20110214030347)
  • Build target: (eg. Darwin_Universal-gcc3)
  • Locale: (eg. en-US)
  • Channel: (eg. nightly)
  • OS version: (eg. Darwin 10.6.0)
  • Distribution: (eg. default)
  • Distribution version: (eg. default)

A client match rule defines criteria for one or more of the above attributes, optionally using a regular expression. One or more client match rule may be associated with any given snippet. When a request URL matches all associated rules, any associated snippets are included in the response. Note that for inclusion rules, all associated inclusion rules must match for the snippet to be included.

The exception is when a rule is flagged as an exclusion rule. In that case, a snippet is excluded from the response when an associated exclusion rule is matched, regardless of any other rules matching.

Service response as a whole

The response from the service is the result of snippets associated with client match rules triggered by the request URL, sorted by the sort order priority.

Keeping this in mind is important in considering what the response to any given client will look like. It means you can do things like:

  • serve up common CSS and JS snippets for all locales
  • include localized HTML content sandwiched between CSS and JS, using the sort order
  • serve up multiple blocks of HTML, all hidden by CSS and selectively revealed by JS at random

Example snippets content

Here's an example JSON export of a set of client match rules and associated snippets:

You can take a look in a text editor, or try importing it into a private dev instance of the service.

In a nutshell, this demonstrates:

  • CSS and JS across all locales to manage content styling and visibility
  • snippets served up to just en-US / en-GB locales, other locales get the default snippets shipped with the browser
  • multiple HTML snippets included, hidden with CSS and one randomly revealed with JS
  • images encoded as data: URIs

QA Process

  1. Snippet content is finalized, including a proofread and link-check before handed off to QA
  2. A "Please test Snippet X" bug is filed and assigned to mozwebqa@mozilla.org in the Other : Snippets : General component in Bugzilla
  3. QA follows the steps on https://wiki.mozilla.org/Firefox/Projects/Firefox_Start/Snippet_Service#Using_a_non-production_service_with_about:home against staging, with the Snippets Switcher add-on installed (en-US only, unless otherwise agreed upon)
  4. QA marks the bug Resolved FIXED when functionality is vetted
  5. Usual deployment process follows (is this documented? Can we document it here?)

Data Specification

As of late 2013, the Snippets service includes a JSON resource intended to serve snippets to Firefox for Android clients. This resource is accessed at a URL like this: https://snippets.allizom.org/json/1/Fennec/28.0a1/20131204030203/Android_arm-eabi-gcc3/en-US/nightly/Linux%2018/default/default/.

This service returns JSON in the following format: 

[
 {"url": "https://mozilla.org", 
 "text": "This is text message for snippet #1. Links to mozilla.org", 
 "target_geo": "US",  # optional. obsolete, replaced by "countries"
 "countries": ["US", "GR"], # optional
 "id": 1, 
 "icon": "data:imag..."
 "weight": 100 },
...
]
Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox/Projects/Firefox_Start/Snippet_Service&oldid=1075935"