CloudServices/Sagrada/Roadmaps/2012Q2

DRAFT Q2 2012 goals

Metlog

• Examine expanding default collection options on metlog messages to meet open standard CEE requirements [COMPLETE]
• Production scale logging deployed for Token, AITC and Queuey [ALMOST]
• Create Node client libs with the same level of functionality as the python ones [NOT DONE]
• Answer at least one product-driven question that shows off the power of Metlog [DEFERRED]
• Start publicizing Metlog to the rest of Mozilla [DEFERRED]
• Draft specifications for and explore implementation ramifications of metlog as external service [NOT DONE]
• Investigate approach for providing metlog / logstash infrastructure for other Mozilla groups and services that will want to use metlog
• Get `vagrant-metlog` fork of `vagrant-centos` set up that will provide a full working metlog->logstash->[statsd,logfiles,etc.] rig ready for client connections [IN PROGRESS]

Queuey

• Preliminary Slabs integration [ON HOLD]
• Launch Queuey for Socorro (assuming hardware dependency resolved) [ON HOLD]
• Finish implementing api for consumer lib and incorporate feedback from external clients [COMPLETE PENDING SOCORRO]
• Metlog integration in queuey and qdo [COMPLETE]

Discovery

• Lock down discovery service syntax

TokenServer

• Beta deployment supporting AITC [COMPLETE]
• Scale up to full version for AITC (and Sync) official launch [COMPLETE]
• Package up so that it's as easy as possible for external users to get up and running [COMPLETE]
• Documentation for public release [COMPLETE]
• C++ crypto workers - Python workers are handling high loads without concern, so this is no longer a goal.
• Add CEF logging

Circus

• Build an interface to control workers from a web console [ONGOING INTERN PROJECT]

Cornice

• Automatic API documentation
• Automated curl-syntax/test generation
• Refactor central libraries

SyncStorage

• Launch AITC [ON TRACK]
• Finish up internal improvements to support 2.0 protocol, including better logic to prevent data collisions [ON TRACK]
• Clean up underlying mysql to better handle transaction management, generation and isolation [DEFERRED TO Q3]
• Make memcache layer less sync-centric and more configurable [ON TRACK]
• Implement a deferred-delete strategy to try to keep those from happening in real time [DEFERRED TO Q3]
• Do a technical analysis and make a call on using changesets for performance [DEFERRED TO Q3]
• Scale out durable storage instances (including aitc) [ON TRACK]
• Implement hibernation plans for inactive users [DEFERRED TO Q3]
• Cleaning up Sync 1.1 tree bitrot [ON TRACK]

HSM

• Plan and implement public and private key distribution [COMPLETE]
• Document signing API [COMPLETE]
• Launch preliminary version in preparation for Marketplace beta [COMPLETE]
• Scale out system for full Marketplace launch [COMPLETE PENDING MARKETPLACE]
• Preliminary plans to support multiple colos [ON HOLD]

CI

• Update build process docs to incorporate new information about CI [ALL DEFERRED DUE TO OPERATIONS CONSTRAINTS]
• Create howto doc/framework for setting up a new project in jenkins
• Enforce pinning of dependencies on tagged builds (or all builds, with a flag to turn it off?)
• Set up Jenkins internal server + public summary page
• Define how ops and devs interact with configuration files
• Deploy one live production server being canary tested