CloudServices/Sync/FxSync/FaviconSync

Bugs

• Old tracking bug: bug 428378
• Initial Sync engine implementation: bug 675299
• Places changes: bug 674238

Goals

• Sync favicons for history and bookmarks
• Don't sync history favicons if only bookmark sync is enabled (viceversa not as problematic)

Coordination

Important to sync up with the Places plan, which involves some schema alterations. mak is keen to reduce space overhead, which will likely see timestamps drop to int32, eliminate or render numeric MIME type values, and store large icons outside of the DB. It would also be nice to unify expiration timestamps and lastModified in the proposed favicon store.

API work should be done in mozIAsyncFavicons, not nsIFaviconService. It's experimental, which means we have more leeway to make changes, and of course it's asynchronous.

Proposal

Firefox (places.sqlite)

moz_places
  id
  guid
  url
  favicon_id  -------> foreign key to moz_favicons.id
  ...

moz_historyvisits
  id
  place_id  ------> foreign key to moz_places.id
  visit_date
  ...

moz_bookmarks
  id
  guid
  type
  fk ------> foreign key to moz_places.id
  ...

moz_favicons
  id
  url
  data
  mime_type
  expiration
  guid (NEW)
  lastmodified (NEW? Integrate with expiration?)

Sync

bookmarks
  id (a.k.a. GUID)
  bmkUri
  ...
  faviconGUID (NEW)

history
  id (a.k.a. GUID)
  histUri
  title
  visits
  faviconGUID (NEW)

favicons (NEW)
  id (a.k.a. GUID)
  url
  data
  mime_type
  expiration?

Implementation

• Implement favicons engine with above data schema.
• Add faviconGUID property to bookmarks and history engines, it's synced as follows:

1. Find favicon based on faviconGUID
   a. it exists. Go to 3.
   b. it doesn't exist. Go to 2.
2. Create empty favicon entry with id and guid.
3. Refer to favicon id in the moz_places entry.

More detailed plan

I propose a staged implementation, given dependencies on Places changes. Something a little like this:

1. Extend bookmarks and history engines to track faviconURI. We do this instead of GUID as a trivial proof of concept, rather than waiting around.
2. Implement favicon engine to sync favicons keyed by URI.
3. Implement observers for nsINavHistoryObserver::onPageChanged/nsINavHistoryObserver::ATTRIBUTE_FAVICON to track bookmark/history changes on favicon alteration, and also poke the favicon tracker to ensure that changed favicons get synced up.
4. Implement GUIDs for favicons, in line with the Places plan.
5. Rejig favicon and other engines to use the GUID instead of the URI. This will, of course, be a breaking change (and we don't want to use URIs in production for privacy reasons), so this will be the first point at which this can be a user-facing feature.
6. Reimplement bookmarks and favicons engines using async repository API.
7. Switch to using mozIAsyncFavicons instead of nsIFaviconService.

Questions/Observations

• Is a blank entry in moz_favicons (cf. step 2 above) ok?
• How should we model the expiration (if at all)?
• How to access favicon data? Raw SQL r+w? Extend mozIAsyncHistory?
  • mozIAsyncFavicons. This API is ours to chop up and extend, with appropriate judgment.
• How can we be notified when a new favicon is added or an existing one is changed?
  • Observers. See above.
• How do we tie into or interact with Places' favicon expiration events?
• It would seem to make sense to sync favicons before bookmarks, so as to have favicons present in the DB before they are linked to their bookmark record. We should also take care to keep toolbar/menu icons very much in the working set: synced first, and generally given priority. UX über alles!
  • I disagree with that. It would be more useful to see people's bookmark toolbar/menu etc. populated with entries and then add the favicons quickly thereafter... Favicons could probably take considerably more time to sync than bookmarks... But as you say, that's something for UX to decide. --philikon
• Can this scheme easily be extended to work for things like site thumbnails?