Overview Architecture Diagram

Diagram

• 

  Source Architecture Overview

• 

  Architecture Overview

• 

  Architecture Overview with tree closure reasons

Tree Closure Reason Codes

1. TBPL or tree herder: issues will close the tree if build_data is >5-30 minutes out of date; or if reads of Bugzilla data fails, tree may close
2. TBPL or tree herder: can't read from pushlog
3. Push to Try tree takes too long: push succeeds, but pushlog isn't updated and so corrupts Try, and/or Try itself can't be updated
4. Can't read, can't start new jobs or no new commits are seen so no new jobs are created
5. Ops issues: failovers of databases require coordination and downtimes so no new jobs are created
6. Buildbot fails to insert new build
7. Puppet config change puts slaves out of service
8. No more machines in a pool, tree closes
9. AWS + Buildbot or AWS + Hg -> network pipe fills up and trees are closed manually by sheriffs who noticed a lot of errors/alerts
10. False negative failures because we can't fetch python libraries or some other necessary tests. Similarly, if gonk can't get updates because bumpers fail or syncs to git.m.o fail and we're stale, sheriffs might close the tree
11. Can't read from git.m.o
12. Product signing fails so some or all trees might be closed. To fix, requires a key phrase to be manually re-entered. Similarly, if symbols upload, test output upload fails, trees can close because output functions aren't working from otherwise successful builds.