ReleaseEngineering/Python Standards

Mozilla Release Engineering Python Standards

Scope

This document describes python usage by Release Engineering infrastructure. This includes things like scriptworkers, services, balrog.

Build and test environments are not in scope here. We aim to develop a similar set of standards for those environments.

Supported versions

Any new projects should use Python 3 by default. Python 2 backwards compatibility is not required.

Projects that are currently using Python 2 must have a plan for migrating to 3.

We will support the latest version of Python 2.7.

We will support the most recent major version of Python 3.x after the 3.x.1 release.

Updates

We will deploy updates to Python 2.7 within one month of release.

We will deploy minor version updates to Python 3.x within one month of release. For example 3.6.4 -> 3.6.5 will be done within one month of 3.6.5 being released.

Will will deploy major version updates to Python 3.x within one month of the 3.x.1 release. For example once 3.7.1 is released, we will have our systems upgraded from 3.6.x to 3.7.1 within a month.

Current versions

Our infrastructure is currently running on 2.7.17 and 3.8.1

Package dependencies

Version pinning

All dependencies of a program / module should be fully specified and pinned, with sha-256 hashes specified as well. These dependencies include direct and transitive dependencies.

Projects should have a mechanism in place to monitor for new upstream dependencies, and run automated tests against the updated dependencies. E.g. https://pyup.io/

setup.py should specify top-level project dependencies, un-pinned. This is intended only for local development. See https://packaging.python.org/discussions/install-requires-vs-requirements/ for more information.

requirements.txt should specify all dependencies, fully pinned with sha-256 hashes. Deployment should use requirements.txt, not setup.py

Package index / repository

We must not rely on the public python package index (https://pypi.python.org/pypi) in production infrastructure. Any packages needed should be fetched via https from a mirror we control (e.g. https://pypi.pub.build.mozilla.org/pub/), or vendored with the project source code.

Style

PEP8 with ~90 character line length. https://www.python.org/dev/peps/pep-0008/. However, readability matters more than following the style guide!

Use flake8-isort to enforce consistent module import order. Standard library imports go first, then third party imports, and finally local imports.

Use flake8-commas to enforce consistent trailing commas.

Use Google style docstrings for documenting function argument and return types https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html. Flake8-docstrings will help enforce this.

Testing

Use tox as a top-level test harness in order to easily test multiple python versions.

Unitests should use pytest with code coverage enabled. Code coverage should be reported to https://codecov.io/.

We should aim for at least 90% code coverage. New contributions to a project should never decrease code coverage.

Tests should enforce style guidelines

Should test with latest 2.7, 3.x and 3.(x+1) if available. Use Travis or Taskcluster for running tests automatically. Tests should run quickly and with no superfluous output.

License

All projects must be licensed under the MPL-2.0 license. https://www.mozilla.org/en-US/MPL/

Repository structure

Top level directories: src, tests