Thunderbird/Infrastructure/TinderboxTLC

Congratulations! You may be the proud new user and/or sheriff of a Mozilla Thunderbird Tinderbox. Here are some important bits of info you should know.

Related Tinderbox Trees

This page primarily relates to the Thunderbird trees: Thunderbird trunk, Thunderbird 3.0, Thunderbird Try

Sheriff

• Due to the low level of commits, and generally stable trees, there isn't normally a sheriff for the Thunderbird trees.
• If there is a sheriff then it will be listed on the appropriate Tinderbox page (There is no sheriff for the Thunderbird Try tinderboxes).
• If in doubt ask on #maildev on irc.mozilla.org.

Uh oh, something broke

Mozilla-* broke the tinderboxes

We pick up the most recent changes from mozilla-central (and appropriate branches). These are tracked on the Firefox tinderboxes.

If there is bustage that looks to be in a mozilla-central part of the build, then its quite possible that it is a mozilla-central bustage. Here's what to do:

• Head over to the appropriate Firefox tinderbox page.
• If the bustage is showing there, then its probably a mozila-central regression, see the instructions there for details.
• If the bustage isn't showing there, there are two possibilites:
  1. it hasn't shown up yet.
  2. a mozilla-central change has broken the Thunderbird builds.

In the latter case, it may be that a bustage fix is required. If the fix is simple, this can normally be checked in stating it is a bustage fix. If the fix is complex, it is normally best to file a bug.

If in doubt, ask in #maildev.

Random Test Failures

Sometimes random failures do happen.

• Frequent random failures known to be specific to Thunderbird are generally listed at the top of the tinderbox pages.
• Known random failures in core (mozilla-central) are listed in the dependencies of bug 438871.
• If the failure isn't listed or already filed, then you should file a bug! A failing test *is always a bug*. It's either a bug in the test or in the code, but it needs to be tracked.

Tinderbox Failures

Sometimes, clearer information can be obtained by looking at the buildbot waterfall page.

Clobber required

The tinderboxes didn't build correctly and need a clobber to try and fix them.

Individual builders, or sets of builds can be clobbered here.

If you don't have access for that page ask in #maildev.

Obviously, clobber builds take longer than depend builds, so please only use this functionality when necessary; also, please coordinate clobbering on IRC, so builds aren't clobbered multiple times for the same reason.

If you're a regular committer and have hg access, file a bug to get access.

Forcing a new build

This can be done by talking to the various thunder* bots on #maildev.

Please only do if absolutely necessary as this will mean the builders will take longer to get to new patches.

That box is broken

Sometimes we get:

• Access denied to remove files
  • Typically caused by a hung process
• Connection Lost
  • Sometimes buildbot looses connections, due to heavy load or network issues, these typically clear the next cycle.

Quite often these are localized to one virtual machine and can therefore appear as intermittent failures on tinderbox, when its actually a permanent failure on that virtual machine, the buildslaves view can help diagnose these.

If you think a builder needs a poke, talk to gozer or Standard8 on #maildev. Standard8 only has access to some VMs.

If they aren't around, then file a bug.