Our Technologies

The following is a non-exhaustive lists of the technologies the team primarily chooses to use. When developing a new system, especially under tight time constraints, it is a good idea to stick with these technologies.

Programming Languages

Python (3.5+)

Python is our team's primary programming language. It is a versatile language which is both easy to learn and powerful. Much of our team's previously developed software is built with Python.

Python 3 is the preferred version as it is the future of Python development and most libraries are compatible.

Notes:

• Python has good enough performance for most tasks and is adequate until measurements show otherwise.
• There are many existing Python services interacting with Mozilla's various systems, which may serve as examples.

The following is a set of python libraries commonly used by our team (It's best to have a concrete reason when considering an alternative library that is not listed here):

• click
• connexion
• Django
• Flask
• Kombu
• pytest
• requests
• SQLalchemy

Python (2.7.x)

Python 2 is used where version 3 is inappropriate. This includes the following locations:

• Mercurial extensions (mercurial's python 3 support is incomplete 2017-06-16)
• Scripts packaged with the Firefox build environment
• Modifications to existing systems built with Python 2

Javascript

Javascript (ES6) is used by our team for building client side web applications.

Supporting Services

uWSGI

uWSGI is our preferred WSGI server/gateway. (Here's why we chose this.)

PostgreSQL

PostgreSQL is our preferred database.

Notes:

• Traditional RDBMS' are well understood by our development team
• Our operations team is more experienced with SQL systems vs newer NOSQL databases
• PostgreSQL can fulfill a similar role to NOSQL databases using the built-in JSON data types.
• There are mature libraries and tools for working with PostgreSQL
• Performance characteristics are well understood and adequate

Redis

Redis is the preferred choice for in-memory storage of data. It performs comparably with memcached in traditional caching scenarios but also supports more powerful data types.

Redis is also suitable as a persistent data store when configured correctly. This is at the expense of some performance, but may be used as such where it makes sense.

Amazon SNS/SQS

Amazon's notification and messaging services are the preferred choice when a reliable internal message queue is required.

RabbitMQ (Mozilla's Pulse)

Messages which are relevant to a wider audience may be published to Mozilla's Pulse service. While it is possible to use Pulse for non-sensitive internal messages, Amazon SNS/SQS is preferred.

Conceptual Choices

OpenAPI (Swagger)

RESTful

No Synchronized Deployments

Conduit is made up of a number of services, some large (BMO, Phabricator), some small (lando-ui, lando-api). Each of these services communicates with at least one of the others. Thus, there are occasions in which modifications to two (or more) services are necessary to fix an issue or implement a new feature.

In these cases, it is important to carefully plan the changes to not require synchronized deployments. Especially in the case of the larger applications, timing deployments can be difficult, and there always exists the possibility that one service will have to be rolled back, further complicating the situation. Instead, changes should be designed to be backwards compatible, such that they can exist on one system before the associated change is rolled out to the other. Versioned APIs would make this easiest, but in other cases simple checks like the absence or presence of a particular key in a request or response is sufficient.

Ideally, in the case of a change to two systems, both will be backwards compatible, but we may be able to get away with only having one be fully compatible if there exists a reasonable certainty that it will be deployed first.