Security/Reviews/CloudServices/Marketplace Backend
Contents
Status
| Identity (TEMPLATE) | |
|---|---|
| Tracker Bug | - |
| Stage | Definition |
| Status | Green (Green, Yellow, Red?) |
| Release Target | |
| Health | - |
| Status Note | - |
Team
| Product manager / Feature manager | Lindsay Saunders, Stephanie Turner and Caitlin Galimidi. |
| Engineering lead | |
| Engineering manager | Payments - Andy McKay
Marketplace - |
| Security lead | Adam Muntner |
| Privacy lead | |
| Localization lead | Peiying Mo |
| Accessibility lead | - |
| QA lead | Krupa Raj |
| UX lead | Elizabeth Hunt |
| Product marketing lead | |
| Additional members | Amy Tsay - Community
Scott DeVaney - Editorial Manager |
Open issues/risks
Stage 1: Definition
Introduction
Marketplace Backend, also known as "Zamboni."
Links
- Mozilla Marketplace wiki page: https://wiki.mozilla.org/Marketplace
- Source code: https://github.com/mozilla/zamboni
- Documentation: https://marketplace.readthedocs.org/en/latest/
- Production site: https://marketplace.firefox.com/
- Testing: https://marketplace.allizom.org
- Global Constants for Marketplace: https://github.com/mozilla/marketplace-constants
Mana pages (Mozilla staff/contrib LDAP account needed)
- Addons/Marketplace Mana Page - https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=9601080
- Marketplace / AMO physical infrastructure - https://mana.mozilla.org/wiki/display/SVCOPS/Marketplace+AMO
Related
- Jugband engineering metrics and activity for Marketplace - https://jugband.paas.allizom.org/
- Firefox Accounts Security Review https://wiki.mozilla.org/Security/Reviews/B2G/FirefoxAccounts
Use Cases
Data Flows
Diagram
Marketplace Backend is in the API Layer:
Diagram of Marketplace and Payments architecture layers by Wil Clouser
Description
1. Section
| ID | Origin | Destination | Description |
| 1.A | Abcdefg hij klmnop | Abcdefg hij klmnop | Abcdefg hij klmnop. |
| 1.B | klmnop | klmnop klmnop | klmnop klmnop klmnop klmnop. klmnop klmnopklmnopklmnop |
2. Section
| ID | Origin | Destination | Description |
| 2.A | Abcdefg hij klmnop | Abcdefg hij klmnop | Abcdefg hij klmnop. |
| 2.B | klmnop | klmnop klmnop | klmnop klmnop klmnop klmnop. klmnop klmnopklmnopklmnop |
3. Section
| ID | Origin | Destination | Description |
| 3.A | Abcdefg hij klmnop | Abcdefg hij klmnop | Abcdefg hij klmnop. |
| 3.B | klmnop | klmnop klmnop | klmnop klmnop klmnop klmnop. klmnop klmnopklmnopklmnop |
Architecture Diagram
Marketplace Services diagram by Wil Clouser
Description
Logical diagram of Marketplace application services architecture.
Usage
The dotted line from a red service goes to a breakout describing the logical components of its service stack. This is a logical diagram, it does not represent the deployed physical network or server architecture.
Starting from a Red Services box,
- Solid line represents dependency/backend component relationships
- Dotted lines point to breakouts describing the logical components of a service stack.
Example
The solid lines show that Redis uses Celery and Webheads.
The "Celery" breakout shows that that RabbitMQ is the backend for Celery.
The RabbitMQ breakout shows it has two dependencies, Celery and Webheads
Mozilla Services
Descriptions and links to services in the Architecture Diagram.
Mozilla Services Glossary - https://docs.services.mozilla.com/glossary.html
| Component Name | Description | Documentation URLs |
|---|---|---|
| Celery | Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready). |
Project - http://www.celeryproject.org/ Known Security Issues - http://www.cvedetails.com/vulnerability-list/vendor_id-11748/Celeryproject.html |
| DB0 | Example | Example |
| DB2 | Example | Example |
| DB3 | Example | Example |
| Elastic Search 0 | Elasticsearch is a search server based on Apache Lucene. It provides a distributed, multitenant-capable full-text search engine with a RESTful web interface and schema-free JSON documents. | Project - http://www.elasticsearch.org/ Marketplace Dockerfile - https://github.com/mozilla/marketplace-env/blob/master/mkt/data/images/elasticsearch/Dockerfile |
| Elastic Search 1 | "" | "" |
| Memcache | Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. |
Project - http://memcached.org/ Memcached Dockerfile - https://github.com/mozilla/marketplace-env/blob/master/mkt/data/images/memcached/Dockerfile |
| Monolith | Mozilla Monolith Aggregator is a project to provide statistic gathering, aggregation, a web-service API and a dashboard. The first consumer of Monolith is the Firefox marketplace. Statistics include amongst others public global page views / hits, application specific downloads and even payment related information. This aggregator part deals with gathering data from multiple sources, bringing them into a common format and storing them. Currently data is stored in MySQL for durable archival and ElasticSearch is used provide the actual data access for the web-service and dashboard. |
Project Documentation - https://mozilla-monolith.readthedocs.org/en/latest/ Sourcecode - https://github.com/mozilla/monolith-aggregator/ |
| Rabbit MQ | RabbitMQ is open source message broker software (sometimes called message-oriented middleware) that implements the Advanced Message Queuing Protocol (AMQP). The RabbitMQ server is written in the Erlang programming language and is built on the Open Telecom Platform framework for clustering and failover. | Project Documentation - http://www.rabbitmq.com/documentation.html |
| Recommendation Engine | Recommendation Engine is based on Frappe, a Django application that provides item recommendations for users. Recommendation Engine was developed by Telefonica Research. | Sourcecode - https://github.com/grafos-ml/frappe Documentation - http://grafos-ml.github.io/frappe/ |
| Redis | Redis is an open source, BSD licensed, advanced key-value cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps and hyperloglogs. | Project - http://redis.io/ Marketplace .conf and Dockerfile - https://github.com/mozilla/marketplace-env/tree/master/mkt/data/images/redis |
| Webheads | Example | Example |
External Services
| Component Name | Description | Documentation URLs |
|---|---|---|
| Firefox Accounts | Example | Documentation - https://wiki.mozilla.org/Identity/Firefox_Accounts
|
| Google Analytics | Example | Example |
| Sentry | Example | Example |
| Solitude | Example | Example |
Stage 2: Design
Threat Model
| ID | Title | Threat | Proposed Mitigations | Threat Agent | Rating | Likelihood | Notes | Impact | Notes |
| 1 | Title text | Threat description | Proposed mitigation. | Threat agents | Rating # | Likelihood # | Notes. | Impact Score # – Impact | Notes. |
| 2 | Title text | Threat description | Proposed mitigation. | Threat agents | Rating # | Likelihood # | Notes. | Impact Score # – Impact | Notes. |
| 3 | Title text | Threat description | Proposed mitigation. | Threat agents | Rating # | Likelihood # | Notes. | Impact Score # – Impact | Notes. |
The Firefox Accounts threat model is inherited by Marketplace Backend.
User Interactions
| ID | Summary | Description |
| 1.A | Summary | Description. |
| 1.B | Summary | Description. |
| 2.A | Summary | Description. |
| 2.B | Summary | Description. |
Client Interactions
| ID | Summary | Description |
| 2.A | Summary | Description. |
Server Interactions
| ID | Summary | Description | Path | Input | Output | CEF | CSRF |
| 3.A | Summary | Description | Path | Input | Output | CEF | CSRF |
| 3.b | Summary | Description | Path | Input | Output | CEF | CSRF |
CEF and CSRF columns indicate wether or not CEF logging or CSRF prevention is required for the interactions
Security Recommendations / Open Issues
| ID | Title | Status | Summary |
| [[1]] | Title | Status(Open/Closed) | Summary. |
| [[2]] | Title | Status(Open/Closed) | Summary. |
CEF Logging Requirements
Business Test Cases
Document application specific test cases here
Privacy Risk Analysis
(Status of and link to privacy review and outcome here)
Stage 3: Planning
Application Security Requirements
Document individual requirements for the application here (e.g. CEF logging, captcha, etc)
Operation Security Requirements
Document network/platform security requirements here (e.g. IDS concerns, firewall changes, system hardening reqs, etc)
Mana Website Creation Form
Critical Security Requirements
Itemize individual security blockers here. Reference components in section AppSec or OpSec subsections. These blockers must be addressed before the product can go live.
Stage 4: Development
Repeatable Security Test Cases
Document individual repeatable security test cases here. Include a reference to the source repo, and documentation that governs how to execute test cases.
Secure Coding Guidelines
Document specific secure coding guidelines to be followed and relate them to specific issues/requirements that are specified; capture bug ids related to those issues.
Code Review Milestones
Table 1 - itemized list of code review milestones {i.e. breakdown of specific components that will be reviewed} Table 2 - list of app components/modules that should trigger additional security review (e.g. auth, csrf, file upload handling, etc)
Stage 5: Release
Application Security Verification
These subsections should contain a list of the steps to be taken, and the status of each activity
Code Review
Automated Security Testing
Manual Security Testing
Operational Security Verification
ArcSight Information
Network Design Security Review
Database Security Review
Platform Security (Hardening & Specific Config Requirements)
Landing Criteria
This should be a table itemizing everything from Stage 3 - Critical Security Requirements, including status. For status Red=Unimplemented,Yellow=implemented,Green=tested and passed?
Stage 6: Post Implementation Review
Production Security Considerations
Document additional/ongoing work for this application (e.g. specific things to watch for in ArcSight, gaming behaviour, etc)
Post Implementation Tasks
Itemize process/kb changes developed from this project (e.g. secure coding guidelines, policy stuff, etc)
Team status notes
| status | notes | |
|---|---|---|
| Products | tbd | - |
| Engineering | tbd | - |
| Engineering | tbd | - |
| Engineering | tbd | - |
| Engineering | tbd | - |
| Engineering | tbd | - |
| Engineering | tbd | - |
| Engineering | tbd | - |
| Engineering | tbd | - |
