User:Mjzffr/Resources/Mozharness
From MozillaWiki
< User:Mjzffr | Resources
Contents
What is mozharness?
- It's a collection of Python modules that make it easy to extend and configure common automation tasks
- (Henrik) It's mostly all configuration driven
- Lives in mozilla-central under testing/mozharness: http://hg.mozilla.org/mozilla-central/file/default/testing/mozharness
- Logging and config management (and much more) are already taken care of for you
- Nice things: cross-platform, platform-specific configs, make it easy to create reproducible environment
How do I write a mozharness script? Some tips
- See Resources below
- Get mozharness without checking out all of mozilla-central (See archiver_client.py below)
- (Henrik) I would still propose to use mozilla-central or inbound if the target is to land the script in tree
- Get your script to find mozharness (especially if your script lives in another repo, not in-tree): e.g. https://gist.github.com/mjzffr/f2c7612d2efd3c0c1304#file-firefox_media_tests_jenkins-py-L30
- Your script will probably consist of a class that extends BaseScript or VirtualenvMixin + config
- Use multiple inheritance to plug-in the behaviour you need, then customize. Some examples:
- VirtualenvMixin - create virtualenvs, manage dependencies
- TestingMixin - obtain firefox binary, tests, symbols
- VCSToolsScript - check out hg or git repositories
- Lots more here: http://hg.mozilla.org/mozilla-central/file/default/testing/mozharness/mozharness
- Be careful: if your mozharness script creates a virtual env, you don't want to run it inside your own virtual env
How do I use mozharness scripts in my Jenkins instance?
- Disclaimer: This is just how we set things up on pf-jenkins. There are other ways.
- Use archiver_client.py to put mozharness on your builders (see below)
- Set an environment variable with the path to mozharness on the builders (in our case MOZHARNESSHOME)
- Jenkins job checks out Github repo that contains our mozharness script (Note: our project will soon move from Github to mozilla-central)
- mozharness script is called in build step
References
- https://wiki.mozilla.org/ReleaseEngineering/Mozharness (includes tutorial and simple examples)
- Ask questions on #releng - jlund, armenzg
- Could also ask on #ateam although we don't own mozharness - most recent users are maja_zf, whimboo, armenzg, and others)
- More examples:
- firefox-ui-tests running in mozmill-ci (Jenkins): http://hg.mozilla.org/mozilla-central/file/default/testing/mozharness/scripts/firefox_ui_tests
- firefox-media-tests running per-commit in buildbot: http://hg.mozilla.org/mozilla-central/file/default/testing/mozharness/scripts/firefox_media_tests_buildbot.py
- for running tests on our Jenkins instance and also taking care of Treeherder submission https://gist.github.com/mjzffr/f2c7612d2efd3c0c1304#file-firefox_media_tests_jenkins-py (Note: we'll eventually remove Treeherder submission from this script because it will be taken care of by TaskCluster)
archiver_client example
1. Download archiver_client: http://hg.mozilla.org/build/tools/file/default/buildfarm/utils/archiver_client.py
2. Run something like `python archiver_client.py mozharness --rev 62cd40885e93` and it will download to ./mozharness
More: https://api.pub.build.mozilla.org/docs/usage/archiver/