Firefox 2.0.0.12:BuildNotes

Build Engineers

Bonsai queries

RC1
- http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=MOZILLA_1_8_BRANCH&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2008-01-28+00%3A01+PST&maxdate=2008-01-28+16%3A00+PST&cvsroot=%2Fcvsroot
- (l10n bonsai link goes here)
RC2
- (based on relbranch)
RC3
- (based on relbranch)

Build data

Type	Build ID	SHA1	Push date	Build machine
[Windows installer/zip]	RC1:			production-pacifica-vm
[Mac compressed]	RC1:			bm-xserve05
[Linux compressed]	RC1:			production-prometheus-vm

Notes

RC1

Bootstrap tag: RELEASE_AUTOMATION_M7

Setup before starting:

used existing version bump bug to attached diff for bootstrap.cfg
- "cvs update" in /home/buildmaster/Automation/bootstrap-configs/ after checkin. Did a cvs stat first to work out if any local patches or if unexpected changes would be pulled in
- restarted master
ensure that machines have enough resources (bug 393274)
- On production-prometheus-vm remove the contents of /builds/updates/* - didn't need to do this, plenty of space
- On bm-xserve05 remove the contents of /builds/verify/*
- On build-console, make sure both private and public areas are synced to stage:
  - rsync -nav /home/ftp/pub/firefox/nightly/2.0.0.11-candidates/ stage.mozilla.org:/home/ftp/pub/firefox/nightly/2.0.0.11-candidates/
  - removed /home/ftp/pub/thunderbird/ after comparing to staging like above
  - rsync -nav /data/cltbld/firefox-2.0.0.11/ stage:/data/cltbld/firefox-2.0.0.11/
    - expect new directories, but file contents should be the same
- On build-console, remove private and public areas:
  - rm -rf /data/cltbld/firefox-*
  - rm -rf /builds/tags/* - did not do this, plenty of space
  - in /home/ftp/pub/firefox/nightly/, remove all except the 2.0.0.x-candidates directory for the previous release
kick off buildbot (run as cltbld):
- buildbot sendchange --master=localhost:9989 -u joduinn -m"Firefox 2.0.0.12 RC1" release

Tag

Source

permissions problems in /home/ftp/pub/firefox/nightly prevented rsync to stage
- to resolve: chmod a-s /home/ftp/pub/firefox/nightly
- need to investigate what's causing this

Build

win32 build failed, CVS_RSH was not set in environment
- buildbot slave was started from cmd.exe not cygwin
- FIXME make this run on boot so we can't get it wrong :)
- removed FIREFOX_2_0_0_12_* tags on tinder-config.pl/mozconfig and re-ran
mac build failed
- removed FIREFOX_2_0_0_12_* tags on tinder-config.pl/mozconfig and re-ran
- intermittent failure seen on staging:

----------- Output from MozillaAliveTest ------------- 
 2008-01-28 22:14:33.998 firefox-bin[9432] CFLog (0): CFMessagePort: bootstrap_register(): failed 1100 (0x44c), port = 0x3d03, name = 'Processes-0.2228225'
 See /usr/include/servers/bootstrap_defs.h for the error codes.
 2008-01-28 22:14:33.999 firefox-bin[9432] CFLog (99): CFMessagePortCreateLocal(): failed to name Mach port (Processes-0.2228225)
 CFMessagePortCreateLocal failed (name = Processes-0.2228225 error = 0)
----------- End Output from MozillaAliveTest ---------

Sign

Signing doc

L10nVerify

Automated verify ran. For linux & mac, worked fine, nothing to do. For win32, failed out with expected error.

Updates to Test Channels (betatest & releasetest)

automated, nothing to do.

update Verify

automated, nothing to do.

Stage

automated, nothing to do.
waiting for QA go/nogo.
bug 414856 declared showstopper.

RC2

Bootstrap tag: RELEASE_AUTOMATION_M7

Setup before starting:

used existing version bump bug to attached diff for bootstrap.cfg
ensure that machines have enough resources (bug 393274)
- On production-prometheus-vm remove the contents of /builds/updates/* - didn't need to do this, plenty of space
- On bm-xserve05 remove the contents of /builds/verify/*
- On build-console, make sure both private and public areas are synced to stage:
  - rsync -nav /home/ftp/pub/firefox/nightly/2.0.0.11-candidates/ stage.mozilla.org:/home/ftp/pub/firefox/nightly/2.0.0.11-candidates/
  - rsync -nav /data/cltbld/firefox-2.0.0.11/ stage:/data/cltbld/firefox-2.0.0.11/
    - expect new directories, but file contents should be the same
- On build-console, remove private and public areas:
  - rm -rf /data/cltbld/firefox-*
  - rm -rf /builds/tags/* - did not do this, plenty of space
  - in /home/ftp/pub/firefox/nightly/, remove all except the 2.0.0.x-candidates directory for the previous release
kick off buildbot (run as cltbld):
- buildbot sendchange --master=localhost:9989 -u joduinn -m"Firefox 2.0.0.12 RC1" release

Tag

tag respin code determined relbranch incorrectly
- fixed and landed in bug 414966

Source

automated, nothing to do.

Build

had to disable part of tag workaround bug 414966 so win32 and mac builds would work
linux, mac builds handed over to QA
unsigned win32 builds were waiting to be signed, when QA discovered that bug 413250 has another exploit path. QA declare "nogo". Stopped waiting to respin.

RC3

Bootstrap tag: RELEASE_AUTOMATION_M7

Setup before starting:

used existing version bump bug to attached diff for bootstrap.cfg
ensure that machines have enough resources (bug 393274)
- On production-prometheus-vm remove the contents of /builds/updates/* - didn't need to do this, plenty of space
- On bm-xserve05 remove the contents of /builds/verify/*
- On production-1.8-master, make sure both private and public areas are synced to stage:
  - rsync -nav /home/ftp/pub/firefox/nightly/2.0.0.11-candidates/ stage.mozilla.org:/home/ftp/pub/firefox/nightly/2.0.0.11-candidates/
  - rsync -nav /data/cltbld/firefox-2.0.0.11/ stage:/data/cltbld/firefox-2.0.0.11/
    - expect new directories, but file contents should be the same
- On production-1.8-master, remove private and public areas:
  - rm -rf /data/cltbld/firefox-*
  - rm -rf /builds/tags/* - did not do this, plenty of space
  - in /home/ftp/pub/firefox/nightly/, remove all except the 2.0.0.x-candidates directory for the previous release
    - rm -rf /home/ftp/pub/firefox/nightly/2008*
    - rm -rf /home/ftp/pub/firefox/nightly/2.0.0.12-candidates
kick off buildbot (run as cltbld):
- buildbot sendchange --master=localhost:9989 -u joduinn -m"Firefox 2.0.0.12 RC3" release

Tag

automated, nothing to do.

Source

automated, nothing to do.

Build

automated, nothing to do.

Sign

Signing doc

L10nVerify

Updates to Test Channels (betatest & releasetest)

Initially failed due to existing /builds/updates/firefox-2.0.0.12/patcher/mozilla, removed /builds/updates/firefox-2.0.0.12/ and restarted.

update Verify

Linux and Mac update verify failed initially, config bumper code problem bug 415261
Win32 should have failed, but warned instead. reopened bug 408157

Stage

Updates to Beta Channel

beta updates getting served completes not partials bug 388524
- fixed this bug, had to regenerate and re-push snippets

verify beta snippets versus betatest snippets

cd /opt/aus2/snippets/staging/20080201-02-Firefox-2.0.0.12-beta/
find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/betatest/; system("diff -r -u $_ ../20080201-02-Firefox-2.0.0.12-test/$a");' 2>&1 | tee /tmp/beta.log

QA declare "nogo". Stopped waiting to respin.

RC4

Bootstrap tag: RELEASE_AUTOMATION_M7

Setup before starting:

used existing version bump bug to attached diff for bootstrap.cfg
attach diff to update verify, remove current version due to bug 415261
ensure that machines have enough resources
- FIXME should be automatically backed up or removed - bug needed
On production-prometheus-vm remove the contents of /builds/updates/
- FIXME need to do this or updates will fail - bug needed
On bm-xserve05, production-pacifica-vm, production-prometheus-vm remove the contents of /builds/verify/*
- FIXME should be automatically backed up or removed - bug needed
On stage, rename ~/firefox-2.0.0.12 to ~/firefox-2.0.0.12-rc3
- FIXME should be automatically backed up or removed - bug needed
On production-1.8-master, remove private and public areas:
- FIXME should be automatically backed up or removed - bug needed
- rm -rf /data/cltbld/firefox-*
- rm -rf /builds/tags/* - did not do this, plenty of space
- in /home/ftp/pub/firefox/nightly/, remove all except the 2.0.0.x-candidates directory for the previous release
  - rm -rf /home/ftp/pub/firefox/nightly/2008*
  - rm -rf /home/ftp/pub/firefox/nightly/2.0.0.12-candidates
kick off buildbot (run as cltbld):
- buildbot sendchange --master=localhost:9989 -u joduinn -m"Firefox 2.0.0.12 RC4" release

Tag

automated, nothing to do.

Source

automated, nothing to do.

Build

automated, nothing to do.

Sign

Signing doc

L10nVerify

automated, nothing to do.

Updates to Test Channels (betatest & releasetest)

PatcherConfig undid the DisableCompleteUpdates change in bug 388524, bug reopened.
- landed fix and re-generated AUS config:

# backup current config
cd /builds/updates/firefox-2.0.0.12/patcher/temp/firefox/2.0.0.11-2.0.0.12
mkdir rc4.1
mv aus2.* rc4.1
# pull in fixed config
cd /builds/updates/firefox-2.0.0.12/config/
cvs up
# regenerate AUS config
cd /builds/updates/firefox-2.0.0.12/patcher/
./patcher2.pl --app=firefox --config=../config/moz18-branch-patcher2.cfg --create-patchinfo 2>&1 | tee patchinfo-rc4_2.log

push new config to AUS

rsync -e ssh -av aus2.beta/ aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20080203-Firefox-2.0.0.12-beta/
rsync -e ssh -av aus2.test/ aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20080203-Firefox-2.0.0.12-test/
rsync -e ssh -av aus2/ aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20080203-Firefox-2.0.0.12/

verify beta snippets versus betatest snippets
- FIXME automate this in bug 409449

cd /opt/aus2/snippets/staging/20080203-Firefox-2.0.0.12-beta/
find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/betatest/; system("diff -r -u $_ ../20080203-Firefox-2.0.0.12-test/$a");' 2>&1 | tee /tmp/beta.log

update Verify

automated, nothing to do.
FIXME - should verify all previous releases, not just the immediately previous bug 408453

Stage

automated, nothing to do.

Updates to Beta Channel

After QA gives "ok" to push to beta channel, on aus2-staging do:

 # put snippets on beta
 $ cd /opt/aus2/snippets/staging
 $ sudo ~cltbld/bin/pushsnip 20080203-Firefox-2.0.0.12-beta

FIXME pushsnip does backup before pushing; we could do this backup in the Updates->Push() method instead. needs bug filed

Sign Installers

NOTE - build-console is no longer used from here on
pull stage:/data/cltbld/firefox-2.0.0.12/batch1/stage-signed/ to signing machine
follow installer-signing-instructions here
push signed bits back to same location on stage
complete stage-merged:

# on stage
cd /data/cltbld/firefox-2.0.0.12/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/

Create MD5 and SHA1 checksum files

# on stage
cd /data/cltbld/firefox-2.0.0.12/stage-merged/
~/bin/checksum-files .

Fix permissions & ownership (on the two SUM files, and the detached sigs)

chown -R cltbld:firefox .
chmod 644 *SUMS

Release

get the last formal "go" from QA/Dev/website/IT/release-drivers
push the stage-merged directory to the releases area:

# on stage
rsync -av /data/cltbld/firefox-2.0.0.12/stage-merged/ /home/ftp/pub/firefox/releases/2.0.0.12/

After pushing live, move forward the "latest" and "latest-2.0" symlinks:

 cd /home/ftp/pub/firefox/releases/
 rm latest
 ln -s 2.0.0.12 latest
 rm latest-2.0
 ln -s 2.0.0.12 latest-2.0

Update stage:/etc/rsyncd-mozilla-current.exclude

Update Bouncer

Add entries to download.mozilla.org

Final Verification

Verify that releasetest points to valid bouncer links:

# this can be run from anywhere
cvs co mozilla/testing/release
cd mozilla/testing/release/updates
cat moz18-firefox-*.cfg | sed 's/betatest/releasetest/' | grep -v 2.0a | grep -v 2.0b > update.cfg
./verify.sh -t update.cfg 2>&1 | tee quickVerify.log

Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):

grep HTTP quickVerify.log | grep -v 200 | grep -v 302

Before pushing final updates,verify that "release" and "releasetest" channel match:

# on aus2-staging
cd /opt/aus2/snippets/staging/20080203-Firefox-2.0.0.12
find -type d -iregex '.*release.*' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20080203-Firefox-2.0.0.12-test/$a");'

Enable update channel

Wait for QA to finish testing update snippets on releasetest channel, then enable update snippets on release channel

 # on aus2-staging
 # put snippets on release/live channel
 $ cd /opt/aus2/snippets/staging
 $ sudo su - cltbld
 $ ~/bin/pushsnip 20080203-Firefox-2.0.0.12

Free Software builds

Bump the BuildTag definition in the three tinder-config.pl, like this

Done on production machines:
- production-pacifica-vm (win32)
- bm-xserve05 (macosx)
- production-prometheus-vm (linux)

Start builds:

cd /builds/tinderbox/Fx-Mozilla1.8-FS
./build-seamonkey.pl --once --depend --config-cvsup-dir `pwd`/tinderbox-configs/ 2>&1 | tee FIREFOX_2_0_0_12_FS.log

Stage

# as cltbld@stage
mkdir -p ~/firefox-2.0.0.12-fs/batch1/prestage
cd ~/firefox-2.0.0.12-fs/batch1
# rsync from push dirs
rsync -av /home/ftp/pub/firefox/nightly/2008-02-07-09-mozilla1.8-fs/ ./prestage/
rsync -av prestage/ prestage-trimmed/
rm -v prestage-trimmed/*.zip
rsync -av prestage-trimmed/ stage/
cd stage
~/bin/groom-files --long="2.0.0.12.fs" .
chown -R cltbld:firefox * .
find . -type f -exec chmod -v 644 {} \;
find . -type d -exec chmod -v 755 {} \;
cd ../
rsync -av stage/ /home/ftp/pub/firefox/releases/2.0.0.12/contrib/free-software/

Module	Tag	Pull date
cvsroot/mozilla	RC1: GECKO181_20080128_RELBRANCH FIREFOX_2_0_0_12_RC1 FIREFOX_2_0_0_12_RELEASE	2008-01-28 16:00 PST
cvsroot/mozilla	RC2: FIREFOX_2_0_0_12_RC2 FIREFOX_2_0_0_12_RELEASE	2008-01-30 13:00 PST
mofo/talkback/fullsoft	RC1: GECKO181_20080128_RELBRANCH FIREFOX_2_0_0_12_RC1	2008-01-28 16:00 PST
l10n/l10n	RC1: GECKO181_20080128_RELBRANCH FIREFOX_2_0_0_12_RC1	2008-01-28 16:00 PST

Firefox 2.0.0.12:BuildNotes

Contents

Build Engineers

Bonsai queries

Tags

Build data

Notes

RC1

Tag

Source

Build

Sign

L10nVerify

Updates to Test Channels (betatest & releasetest)

update Verify

Stage

RC2

Tag

Source

Build

RC3

Tag

Source

Build

Sign

L10nVerify

Updates to Test Channels (betatest & releasetest)

update Verify

Stage

Updates to Beta Channel

RC4

Tag

Source

Build

Sign

L10nVerify

Updates to Test Channels (betatest & releasetest)

update Verify

Stage

Updates to Beta Channel

Sign Installers

Release

Update Bouncer

Final Verification

Enable update channel

Free Software builds

Wall clock timings

Navigation menu

Search