Services/Sync/Protocol 2.0

Status

Sync Storage Protocol 2.0
Stage	Draft
Status	`
Release target	`
Health	OK
Status note	`

Team

Product manager	`
Directly Responsible Individual	`
Lead engineer	`
Security lead	`
Privacy lead	`
Localization lead	`
Accessibility lead	`
QA lead	`
UX lead	`
Product marketing lead	`
Operations lead	`
Additional members	`

Open issues/risks

`

Stage 1: Definition

1. Feature overview

A new Sync HTTP storage protocol will be defined and implemented. The new protocol will address known problems with the existing protocol specification, remove old features, modify behavior, and add new APIs.

2. Users & use cases

`

3. Dependencies

`

4. Requirements

`

Non-goals

`

Stage 2: Design

5. Functional specification

Random wishlist items:

• Sessions. The client has a concept of beginning and ending a sync, with some expectation of consistency within a session -- for example, that nobody else will be racing to write to meta/global while we upload items. Client behavior could be simplified (e.g., eliminating race-reduction handling) and made more robust if sessions were explicit.
  • Related - moving away from millisecond timestamp precision

• Cross-collection operations.

• URL space separations. Consider whether we should re-partition the URL space to allow for operations such as "delete everything apart from meta/global", "wipe all collections apart from keys", or "kill all dependent storage".

• Ability to send one-time messages to clients (such as "this version is being deprecated. Please upgrade")

• Additional information included with X-Weave-Backoff

• Rate-limiting clients who send too-large POST and PUT bodies. 413 with a header.

• If we don't already do it, we should take advantage of more opportunities to send less data: for example, sending 304 Not Modified in response to info and meta requests. In a world of instant syncs, this might avoid schlepping a non-trivial number of bytes over the wire, and some client parsing too.

• Ability to tell client that it isn't supported and needs to upgrade to a new version or switch to a new version of the protocol.

The following API additions are proposed:

The following API removals are proposed:

• Remove use of HTTP 400. While 400 has been overloaded by many, there are still some practical considerations for not using it. Some HTTP stacks and agents (e.g. load balancers) will kill the underlying connection if a 400 is seen. Error handling should be accomplished some other way.

The following API clarifications are needed:

• Define relationship between X-Weave-Backoff and Retry-After HTTP response headers.

See also: Labs/Weave/Sync/1.1/2.0changes

6. User experience design

`

Stage 3: Planning

7. Implementation plan

`

8. Reviews

Security review

`

Privacy review

`

Localization review

`

Accessibility

`

Quality Assurance review

`

Operations review

`

Stage 4: Development

9. Implementation

`

Stage 5: Release

10. Landing criteria

`

Feature details

Priority	Unprioritized
Rank	999
Theme / Goal	`
Roadmap	`
Secondary roadmap	`
Feature list	`
Project	`
Engineering team	`

Team status notes

	status	notes
Products	`	`
Engineering	`	`
Security	`	`
Privacy	`	`
Localization	`	`
Accessibility	`	`
Quality assurance	`	`
User experience	`	`
Product marketing	`	`
Operations	`	`