QA/Platform/Graphics/Guides

From MozillaWiki
< QA‎ | Platform‎ | Graphics
Jump to: navigation, search

Finding a Regression Window

How Mozregression Works

Mozregression works by downloading Firefox Nightly builds from a range of dates allowing you to easily narrow down the first Nightly which introduced a bug. Once a Nightly is downloaded and installed, Mozregression will start Firefox with a temporary profile and allow you to test your bug. After testing the bug, you give Mozregression a good or bad command and it will split the range in half, downloading the next build. Once the range has been narrowed down to a single day it will give you a pushlog for the build that will identify the changes which may have caused the bug. This should be attached to the bug report.

As mentioned above, the range will first be narrowed down to a particular Nightly build, then continue to an Inbound build, which will often get us down to a single patch. After the Nightly build search is done, mozregression will note the range for it, then continue to bisect the Inbound builds. When recording the results of mozregression, it is useful to record the Nightly range as well, even if the Inbound range is found.

Using Mozregression

Note: We recommend narrowing down the range manually before using mozregression.

Using mozregression is as simple as running mozregression from a command prompt. However, this will start with Nightly builds from January 1, 2009 which could make finding the regression range a longer process. Assuming you know the build dates for the first bad and last good Nightly you would run mozgression --good YYYY-MM-DD --bad YYYY-MM-DD where YYYY-MM-DD is the date of each build. See narrowing down the range for information on getting those dates.

1) Open a command prompt and run mozgression --good YYYY-MM-DD --bad YYYY-MM-DD where YYYY-MM-DD is the range of dates you want to test.

2) Mozregression will download builds between those dates, start with a new profile, and allow you to test for your bug.

3) Once you're done testing, switch back to the command prompt and mark the status of the build:

  • bad if the build reproduces the bug
  • good if the build does not reproduce the bug
  • skip if you want to skip testing the bug in that build
  • broken if the build does not work and you're unable to test your bug

4) Mozregression will bisect the range and grab the next nightly, repeating the process above

5) Once the range has been narrowed to a single day it will provide changeset IDs and a pushlog for the range.

  • Make note of this and copy it to your bug report.

6) Mozregression will then begin bisecting the range with mozilla-inbound builds

  • You can skip this step unless someone has asked you to bisect the range further.

Narrowing the Range

Note: We recommend narrowing down the range to six weeks between builds before using mozregression.

1) Start by getting the Build ID for the Firefox build with which you're reproducing the bug

  • Open the about:support page
  • Scroll down to the Application Basics section
  • Make note of the Build ID as the "first bad" build (it's a string of numbers in YYYYMMDDhhmmss format)

2) Install a build from six weeks before that date

3) If that build reproduces the bug

  • Make note of the Build ID as the "first bad" build
  • Repeat step 2 going back another six weeks

4) Once you find a build that does not reproduce the bug

  • Make note of the Build ID as the "last good" build
  • You can now run mozregression and give it the good/bad dates based on the build IDs you noted

For example, if the build ID of a good build is 20150401123456 and the build ID of a bad build is 20150410123456: 

mozregression --good 2015-04-01 --bad 2015-04-10

If Mozregression Fails

If mozregression fails for whatever reason, you'll need to narrow the range manually to a single day:

  • Get the date of the build where you first saw the bug
  • Grab a build that is six weeks older from ftp.mozilla.org
  • Test that build and see if the bug reproduces
  • Repeat this process until you find a build that does not reproduce the bug

Once you've found a range, begin to narrow the range further:

  • Begin by testing the build which represents the midway point.
  • Eg. if the good build is from 2015-04-01 and my bad build is from 2015-05-15, I might test the build from 2015-04-21. The range will be reduced to either 2015-04-21:2015-05-15 or 2015-04-01:2015-04-21, depending on whether or not the bug reproduced.
  • Split the difference again and retest
  • Keep repeating until you've narrowed it down to a single day

Once you find the single day range, get the changeset for each build:

  • Start the Firefox build
  • Open the about:buildconfig page
  • Look at the Source section of this page for an hg.mozilla.org URL
  • Copy the 12-digit ID from the end of that URL (this is the changeset ID for the build)
  • Eg. the changeset of a build built from https://hg.mozilla.org/mozilla-central/rev/4fe763cbe196 is 4fe763cbe196

Once you have both changesets, manually put together the pushlog URL:

Once an offending bug has been identified, that bug should be flagged:

  • mark your bug as blocking the offending bug
  • flag the developer of the offending bug using the needinfo? flag

It may be necessary to bisect the range further but that's outside the scope of this document.

Testing a Try Build

TO BE DETERMINED

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Platform/Graphics/Guides&oldid=1108344"