Security/Reviews/CloudServices/Marketplace Backend

From MozillaWiki
Jump to: navigation, search

Status

Identity (TEMPLATE)
Tracker Bug -
Stage Definition
Status Green (Green, Yellow, Red?)
Release Target
Health -
Status Note -

Team

Product manager / Feature manager Lindsay Saunders, Stephanie Turner and Caitlin Galimidi.
Engineering lead
Engineering manager Payments - Andy McKay

Marketplace -

Security lead Adam Muntner
Privacy lead
Localization lead Peiying Mo
Accessibility lead -
QA lead Krupa Raj
UX lead Elizabeth Hunt
Product marketing lead
Additional members Amy Tsay - Community

Scott DeVaney - Editorial Manager

Open issues/risks

Stage 1: Definition

Introduction

Marketplace Backend, also known as "Zamboni."


Links

Mana pages (Mozilla staff/contrib LDAP account needed)

Related

Use Cases

Data Flows

Diagram

Marketplace Backend is in the API Layer:

Mkt layers.png
Diagram of Marketplace and Payments architecture layers by Wil Clouser

Description

1. Section

ID Origin Destination Description
1.A Abcdefg hij klmnop Abcdefg hij klmnop Abcdefg hij klmnop.
1.B klmnop klmnop klmnop klmnop klmnop klmnop klmnop. klmnop klmnopklmnopklmnop

2. Section

ID Origin Destination Description
2.A Abcdefg hij klmnop Abcdefg hij klmnop Abcdefg hij klmnop.
2.B klmnop klmnop klmnop klmnop klmnop klmnop klmnop. klmnop klmnopklmnopklmnop

3. Section

ID Origin Destination Description
3.A Abcdefg hij klmnop Abcdefg hij klmnop Abcdefg hij klmnop.
3.B klmnop klmnop klmnop klmnop klmnop klmnop klmnop. klmnop klmnopklmnopklmnop

Architecture Diagram

Marketplace Services diagram -  by Wil Clouser Marketplace Services diagram by Wil Clouser

Description

Logical diagram of Marketplace application services architecture.

Usage

The dotted line from a red service goes to a breakout describing the logical components of its service stack. This is a logical diagram, it does not represent the deployed physical network or server architecture.

Starting from a Red Services box,

  • Solid line represents dependency/backend component relationships
  • Dotted lines point to breakouts describing the logical components of a service stack.

Example
The solid lines show that Redis uses Celery and Webheads.
The "Celery" breakout shows that that RabbitMQ is the backend for Celery.
The RabbitMQ breakout shows it has two dependencies, Celery and Webheads

Mozilla Services

Descriptions and links to services in the Architecture Diagram.

Mozilla Services Glossary - https://docs.services.mozilla.com/glossary.html

Component Name Description Documentation URLs
Celery Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).
Project - http://www.celeryproject.org/

Known Security Issues - http://www.cvedetails.com/vulnerability-list/vendor_id-11748/Celeryproject.html

DB0 Example Example
DB2 Example Example
DB3 Example Example
Elastic Search 0 Elasticsearch is a search server based on Apache Lucene. It provides a distributed, multitenant-capable full-text search engine with a RESTful web interface and schema-free JSON documents. Project - http://www.elasticsearch.org/

Marketplace Dockerfile - https://github.com/mozilla/marketplace-env/blob/master/mkt/data/images/elasticsearch/Dockerfile
Known Security Issues - http://www.cvedetails.com/vulnerability-list/vendor_id-13554/product_id-28218/Elasticsearch-Elasticsearch.html

Elastic Search 1 "" ""
Memcache Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.
Project - http://memcached.org/

Memcached Dockerfile - https://github.com/mozilla/marketplace-env/blob/master/mkt/data/images/memcached/Dockerfile

Monolith Mozilla Monolith Aggregator is a project to provide statistic gathering, aggregation, a web-service API and a dashboard.
The first consumer of Monolith is the Firefox marketplace. Statistics include amongst others public global page views / hits, application specific downloads and even payment related information.
This aggregator part deals with gathering data from multiple sources, bringing them into a common format and storing them.
Currently data is stored in MySQL for durable archival and ElasticSearch is used provide the actual data access for the web-service and dashboard.
Project Documentation - https://mozilla-monolith.readthedocs.org/en/latest/

Sourcecode - https://github.com/mozilla/monolith-aggregator/

Rabbit MQ RabbitMQ is open source message broker software (sometimes called message-oriented middleware) that implements the Advanced Message Queuing Protocol (AMQP). The RabbitMQ server is written in the Erlang programming language and is built on the Open Telecom Platform framework for clustering and failover. Project Documentation - http://www.rabbitmq.com/documentation.html
Recommendation Engine Recommendation Engine is based on Frappe, a Django application that provides item recommendations for users. Recommendation Engine was developed by Telefonica Research. Sourcecode - https://github.com/grafos-ml/frappe

Documentation - http://grafos-ml.github.io/frappe/
Top-level project tracking bug - https://bugzilla.mozilla.org/show_bug.cgi?id=1053597
Secreview Bug - https://bugzilla.mozilla.org/show_bug.cgi?id=1045901

Redis Redis is an open source, BSD licensed, advanced key-value cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps and hyperloglogs. Project - http://redis.io/

Marketplace .conf and Dockerfile - https://github.com/mozilla/marketplace-env/tree/master/mkt/data/images/redis

Webheads Example Example
External Services
Component Name Description Documentation URLs
Firefox Accounts Example Documentation - https://wiki.mozilla.org/Identity/Firefox_Accounts


Security Review - https://wiki.mozilla.org/Security/Reviews/B2G/FirefoxAccounts

Google Analytics Example Example
Sentry Example Example
Solitude Example Example

Stage 2: Design

Threat Model

ID Title Threat Proposed Mitigations Threat Agent Rating Likelihood Notes Impact Notes
1 Title text Threat description Proposed mitigation. Threat agents Rating # Likelihood # Notes. Impact Score # – Impact Notes.
2 Title text Threat description Proposed mitigation. Threat agents Rating # Likelihood # Notes. Impact Score # – Impact Notes.
3 Title text Threat description Proposed mitigation. Threat agents Rating # Likelihood # Notes. Impact Score # – Impact Notes.

The Firefox Accounts threat model is inherited by Marketplace Backend.

File:TEMPLATE-Threat-Model.png
TEMPLATE Implementation Dataflow

User Interactions

ID Summary Description
1.A Summary Description.
1.B Summary Description.
2.A Summary Description.
2.B Summary Description.

Client Interactions

ID Summary Description
2.A Summary Description.

Server Interactions

ID Summary Description Path Input Output CEF CSRF
3.A Summary Description Path Input Output CEF CSRF
3.b Summary Description Path Input Output CEF CSRF

CEF and CSRF columns indicate wether or not CEF logging or CSRF prevention is required for the interactions

Security Recommendations / Open Issues

ID Title Status Summary
[[1]] Title Status(Open/Closed) Summary.
[[2]] Title Status(Open/Closed) Summary.

CEF Logging Requirements

Business Test Cases

Document application specific test cases here

Privacy Risk Analysis

(Status of and link to privacy review and outcome here)

Stage 3: Planning

Application Security Requirements

Document individual requirements for the application here (e.g. CEF logging, captcha, etc)

Operation Security Requirements

Document network/platform security requirements here (e.g. IDS concerns, firewall changes, system hardening reqs, etc)

Mana Website Creation Form

Critical Security Requirements

Itemize individual security blockers here. Reference components in section AppSec or OpSec subsections. These blockers must be addressed before the product can go live.

Stage 4: Development

Repeatable Security Test Cases

Document individual repeatable security test cases here. Include a reference to the source repo, and documentation that governs how to execute test cases.

Secure Coding Guidelines

Document specific secure coding guidelines to be followed and relate them to specific issues/requirements that are specified; capture bug ids related to those issues.

Code Review Milestones

Table 1 - itemized list of code review milestones {i.e. breakdown of specific components that will be reviewed} Table 2 - list of app components/modules that should trigger additional security review (e.g. auth, csrf, file upload handling, etc)

Stage 5: Release

Application Security Verification

These subsections should contain a list of the steps to be taken, and the status of each activity

Code Review

Automated Security Testing

Manual Security Testing

Operational Security Verification

ArcSight Information

Network Design Security Review

Database Security Review

Platform Security (Hardening & Specific Config Requirements)

Landing Criteria

This should be a table itemizing everything from Stage 3 - Critical Security Requirements, including status. For status Red=Unimplemented,Yellow=implemented,Green=tested and passed?

Stage 6: Post Implementation Review

Production Security Considerations

Document additional/ongoing work for this application (e.g. specific things to watch for in ArcSight, gaming behaviour, etc)

Post Implementation Tasks

Itemize process/kb changes developed from this project (e.g. secure coding guidelines, policy stuff, etc)


Team status notes

status notes
Products tbd -
Engineering tbd -
Engineering tbd -
Engineering tbd -
Engineering tbd -
Engineering tbd -
Engineering tbd -
Engineering tbd -
Engineering tbd -