Identity/AttachedServices/DeploymentPlanning

From MozillaWiki
Jump to: navigation, search

Overview

Herein we will construct a solid plan for deployment of the various Mozilla-hosted services that make up PiCL aka "New Sync".

Details on the overall product plan are available at User_Services/Sync. From a server-side perspective it consists of four mostly-independent services. We can work on deployment of each independently, but they all need to be standing before New Sync goes into production.

  • Storage Service: This service provides storage of encrypted blobs of data, and is used as the backend storage for New Firefox Sync. Users authenticate to it using BrowserID assertions issued by the Firefox Accounts Service.

Goals and Milestones

The goal for Q32013 is to have the authentication pieces Production Ready.

This does not mean having a fully-deployed production environment! With the implementation of the storage component still outstanding, there's no point in standing up an authentication service all by itself. It does mean that we need the ability to do automated deployments that pass loadtests, meet operational and security criteria, and generally inspire confidence that we could roll things out to production without major hiccups.

An analysis of the expected traffic to the services is available at TrafficModel

These are the top-level milestones on our way to said goal, broken down into weekly chunks:

  • Aug 09:
    • usable manual-deploy dev environment tooling for Firefox Accounts and Scrypt Helper.
  • Aug 16:
    • defined testable "success criteria" for Firefox Accounts deployment (req/sec etc).
    • defined testable "success criteria" for Scrypt Helper deployment (req/sec etc).
    • loadtesting code written and debugged for Firefox Accounts.
  • Aug 23:
    • automated single-region loadtest deployment of Firefox Accounts.
    • loadtests run against Firefox Accounts deployment.
  • Aug 30:
    • fixed any load-related issues in Firefox Accounts.
    • loadtesting code written and debugged for Scrypt Helper and FxA Bridge.
  • Sep 6:
    • automated single-region loadtest deployment of Scrypt Helper and FxA Bridge.
    • loadtests run against Scrypt Helper and FxA Bridge deployments.
  • Sep 13:
    • automated two-region loadtest deployment of Firefox Accounts.
    • loadtests run against Firefox Accounts deployment.
  • Sep 20:
    • fixed any load-related issues in both services.
    • a bit of slack time to account for inevitable slippage.
  • Sep 27:
    • security review signoff for both services.
    • svcops signoff for both services.
  • Sep 30:
    • Production Ready!

And one stretch goal, to work on as opportunity presents itself:

  • Sep 30:
    • auto-updating dev deployments for all these components.

See the sub-pages for each service for more detailed plans, dependencies, etc.

Some things to note about this plan:

  • There are no specific milestones for the Storage Service. It's not defined enough. That may have to change depending on how client work progresses.
  • We're not taking any dependencies on the client-side work. Assuming the protocol stays stable, we'll have the servers ready regardless of whether there's client code landed in Firefox.

Deployment Environment

We plan to deploy all services into AWS.

During initial development we will work with two deployment environments: "dev" and "loadtest". Developers do all their work in the dev environment, spinning up and tearing down small testing stacks at will. Dev and QA collaborate to manage a more fully-fledged loadtest environment that approximates what the service might look like in production. All deployments will live in the "moz-svc-dev" AWS account.

Longer term, we will have dedicated "stage" and "prod" environments managed by SvcOps, in the "mozilla" AWS account.

Common Tooling

We will use the following across all projects and all deployment environments:

Domain Names

Note: entirely made-up and provisional, but you gotta start somewhere. We should talk with SvcOps about using mozaws.net and whatever other standard domains they hang off of.

Each of the three services will have a single canonical endpoint URL, and they will all be under a common domain suffix. Individual deployment stacks will be identified by the common domain suffice.

Dev deployments will have:

  • idp.<stack-name>.lcip.org
  • accounts.<stack-name>.lcip.org
  • scrypt.<stack-name>.lcip.org
  • storage.<stack-name>.lcip.org

The default stack name is "dev" but developers may spin up additional deployment for e.g. testing feature branches.

The loadtest deployment will have:

  • idp.loadtest.lcip.org
  • accounts.loadtest.lcip.org
  • scrypt.loadtest.lcip.org
  • storage.loadtest.lcip.org


Staging and Production environments are not year on the roadmap, and so have not been allocated domains.


Dev Deployment Environment

Managed entirely by dev, for testing or experiments or whatever.

Individual deployments hang off of <stack-name>.lcip.org where <stack-name> may be e.g. the name of a particular feature branch.

The stack "dev.lcip.org" will (eventually) be auto-deployed with the latest version of the code for each project. Probably we can even run automated tests against it. This is not a priority though.

Using awsbox as much as possible, for simple deployments.

Using awsboxen for things that are a little more complicated, e.g. if you want to do a testing deployment of a production-like stack.

Loadtest Deployment Environment

XXX TODO: fill this in as we nail down the details:

  • externally visible domains
  • VPC and subnet setup
  • Logging setup

Open Questions

Here are the TODOs from various sub-projects, all collected into one master list. This needs to reach Inbox Zero ASAP.

  • Who will be in charge of this from SvcOps?
    • mmayo and lloyd to figure out resource allocation
  • Additional "success metrics" like MTTR etc.
  • Budget and costs of running all this stuff.
  • Protocol details to be finalised:
    • storage service
    • scrypt helper
  • detailed instructions for spinning up dev/stage/prod stacks, especially important for QA
Retrieved from "https://wiki.mozilla.org/index.php?title=Identity/AttachedServices/DeploymentPlanning&oldid=699688"