Cloud Services/FirefoxOS Sync

Objective

Meta Bug

This is a proposal to create a framework on FirefoxOS to support apps that wish to synchronize data with a remote server. The framework itself would not provide a means of performing sync; it would not be a general-purpose "sync engine". Rather, it would permit apps to request to be awakened in the background to sync data periodically. When conditions were suitable (e.g., wifi were available, system load were low), apps would be awakened in turn and given an opportunity to sync. An app awakened for background sync should be given some protection by the system, such as a better OOM-kill score, to increase the likelihood that it would complete successfully.

The precise meaning of "sync" could vary from app to app. It could be simple remote backup, or it could be a full transactional two-way sync with conflict resolution, de-duplication, etc.

Several proposed standards and work in progress on FirefoxOS would support this system, in particular:

• BackgroundSync
• ServiceWorkers (which are a requirement for background sync)
  • Offline status (references to SW status)
  • nsm's blog on status of ServiceWorkers
• system messages to wake apps on DataStore change

Architecture

Periodic sync

1. The App is installed with permission to read/write data type T. It wants to sync the T DataStore with a remote service. So its ServiceWorker sends a requestSync message, specifying the duration of the interval between sync requests.

2. The App also wants to watch T for changes, so its ServiceWorker registers a listener with the DataStore, asking to be notified on any changes. (See one-time sync below.)

3. Some time in the future, the Scheduler perceives that it is time for the App to sync. It enqueues the App for the next available sync opportunity (e.g., when the network is available).

4. When it is a good time to sync, the Scheduler sends a sync message to the App's ServiceWorker. This awakens the worker, but does not display the app. The app can now pull and push. (The app is responsible for the sync logic.)

One-time sync

1. Some other App modifies the data in DataStore T. The App's ServiceWorker receives a change notification (see 2 above).

2. The notification wakes the App up in the background.

3. The ServiceWorker sends a non-repeating, immediate requestSync.

4. The app shuts down.

5. The Scheduler receives the sync request. At the next opportunity, it wakes the App with a sync message, and the App syncs the changed data.

It is quite likely that the App in question would be modifying the DataStore at some time, and therefore trigger a change event. It might simply ignore this. However, it could still use a one-time requestSync to allow the Scheduler to decide whether conditions were appropriate for sync. This might be useful in cases where the App was unaware of network status or other relevant parameters (e.g., wifi availability). (More on these below.)

Use Cases

• install a flickr app, and your photos get uploaded to flickr ("sync")
• install a Google+ app, and your photos also get uploaded. Do both at once!
• install Delicious app, sync bookmarks (pending a bookmark DOM API, which we don't have)
• Facebook keeps its contacts in sync.
• Fruux backup keeps contacts in sync, EXCEPT Facebook contacts

Features

User Configuration

Because, apart from the activities of the Scheduler and attached app launcher, the System App is unaware of what data stores are being synced by what apps, configuration of sync and specification of suitable conditions for syncing must be managed in an App's own settings. It is possible that the System Settings could also aggregate a list of apps that have participated in sync by calling requestSync at least once.

The user ought to have some way to specify the right conditions for background sync for each app. Some important conditions might be:

• Whether wifi is available (e.g., for photos and other large data types)
• Whether a given carrier network is available (perhaps for specific Contacts DataStores)
• Whether any network is available
• Whether system load is light
• Whether the battery is well-charged
• Whether it is a time of day when bandwidth is cheaper
• Whether the user wants to disable sync entirely for the app, or otherwise put some conditions on sync; ie don;t sync my work calendar on weekends, etc.

The Network Stats 2.0 Proposal seems helpful here.

Scheduling

The Scheduler is an alarm service that will cause apps to wake up by sending them a synchronization event message. There are two types of requestSync messages the Scheduler should expect, repeating and non-repeating.

Each app should have no more than one series of repeating requests, and no more than one pending non-repeating request.

If an app is uninstalled, the Scheduler must know to forget about it. Likewise, it must be removed from any system Settings lists.

The Scheduler's primary function is to send a synchronization event message to an App to wake it up so that it will know to being syncing (however the App thinks of syncing; the Schedule does not care). The Scheduler must respect the user's configuration preferences for awakening an app. In addition, an awakened app should have some additional assurances: For instance, it should be safer from OOM reaping than other backgrounded apps.

Open questions:

• Should the App be given a limited amount of time to operate (say, 10 minutes)? What would happen with apps that did not complete in that time?
• Should the Scheduler receive feedback if sync succeeds or fails or never returns? Should it re-enqueue failed or lost jobs? Should it notify the user after x failures that the App has not successfully synced? As every App might want this, it seems like a nice feature to offer.

Precedent: Alarm API

• Doesn't account for cases where you don't want the app to be woken up (e.g., the e-mail app shouldn't check for new messages if the phone's battery is running low).
• Can't wake up the app when an external event happens (e.g., you're on Wi-Fi). Scheduling becomes interesting, too; 10 apps shouldn't be woken up simultaneously.

UX Challenges

• How does the user know this is going on in the background?
• How can the user know about resource usage per app?
• If I'm on a limited data plan, there should be a way for me to know that there's a crappy app that's hoovering up all my data (existing network stats? per-app data usage?)
• Dovetailing with success/failure feedback - something like android's notifications that "15 photos synced three minutes ago" - toaster pops down and tells you the results. simple, unobtrusive, non-blocking affordances to tell you that things worked.
• If there's a sync app, where do users find it? (pre-installed, probably) but not a certified app, so it can be updated. it would show up as an update, as play services on android - you can't open that app, but you can update it out-of-cycle
• Would be nice to identify such apps as special in some way - they don't have a launcher, etc. (something in the manifest for this already? "role"?)
• Services app could be bundled with various other apps?