Releases/Firefox 3.6.10/BuildNotes
From MozillaWiki
< Releases | Firefox 3.6.10
Contents
- 1 Notes About Releasing
- 2 Build Engineers
- 3 Signed-off Revision(s)
- 4 Tags
- 5 Build data
- 5.1 Build 1
- 5.1.1 Tag
- 5.1.2 Source
- 5.1.3 Build/Repack
- 5.1.4 Return slaves
- 5.1.5 XULRunner Builds
- 5.1.6 Partner Repacks
- 5.1.7 Unittests
- 5.1.8 Sign
- 5.1.9 L10N Verify
- 5.1.10 Updates
- 5.1.11 Update Bouncer
- 5.1.12 EU Ballot Build
- 5.1.13 Virus Scan
- 5.1.14 Final Checks Before Push
- 5.1.15 Push to mirrors
- 5.1.16 Final verification (sanity check releasetest snippets)
- 5.1.17 Push release/beta channel snippets
- 5.1.18 Push throttling changes
- 5.1.19 Release
- 5.1.20 Push EU Ballot build
- 5.1.21 Push 3.0.19 -> 3.6.10 MU snippets (beta and release)
- 5.1.22 Push XULRunner to mirrors
- 5.1.23 Update XULRunner entries on wiki
- 5.1 Build 1
Notes About Releasing
Please update the Release:Primer for future releases (bug fixes, changes to automation) as needed
Build Engineers
bhearsum, nthomas
Tracking bug
Signed-off Revision(s)
Build1: [1]
Tags
| Build # | Tag | Changeset |
| 1 | FIREFOX_3_6_10_BUILD1 FIREFOX_3_6_10_RELEASE | [2] |
Build data
| Build # | Type | Build ID | Build machine | Time to build |
| 1 | Linux | 20100914123505 | linux-ix-slave17 | 3h7min |
| Mac | 20100914120618 | bm-xserve17 | 1h38min | |
| Windows | 20100914125854 | mw32-ix-slave06 | 1h42min |
Build 1
Done on pm01:8011
- Moved mv-moz2-linux-ix-slave10 to the master.
- Manually clobbered because I wasn't sure how clobberer would behave on a new master:
cd /builds/slave rm -rf updates/* xulrunner_*/* linux_*/* tag/* source/* bouncer_submitter/* partner_repack/*
- Landed automation config
- Deleted /builds/buildbot/moz2-master2/*.pyc
- Reconfiged master
- Closed Firefox3.6 tree
- Started automation:
buildbot sendchange --username=bhearsum --master=localhost:9011 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.10 build1" Go
Tag
- While tagging was happening, moved more slaves to the master, clobbering the release directories before attaching them:
- Linux (mv-moz2-linux-ix-slave17, linux-ix-slave17)
- Mac (bm-xserve17)
- Windows (mw32-ix-slave06)
- Manually tagged buildbot-configs, buildbotcustom, tools with FIREFOX_3_6_10_RELEASE and FIREFOX_3_6_10_BUILD1
Source
No problems with Firefox or XULRunner source generation.
Build/Repack
- Moved more slaves while the en-US builds happened:
- Linux (mv-moz2-linux-ix-slave03 and 04)
- Mac (bm-xserve06 and 07)
- Win32 (mw32-ix-slave12 and 17)
Results:
- Linux
- Slave (one of the latest ix machines) seemed sluggish compared to the older ix machines. Filed bug 596366 on this.
- Mac
- no en-US problems
- some repacks failed out
- Windows
- no en-US problems
- some repacks failed out
Rekicked repacks with the following:
foo-ix-blah:buildbot-helpers bhearsum$ cat shipped-locales is osx rm osx vi osx th osx sr osx hi-IN osx el win32 eo win32 af win32 foo-ix-blah:buildbot-helpers bhearsum$ python force_release_l10n.py -m http://production-master.build.mozilla.org:9010 -t FIREFOX_3_6_10_RELEASE -v -s shipped-locales -n bhearsum
Verified that we had the right number of builds per platform with:
[ffxbld@surf build1]$ cd /home/ftp/pub/firefox/nightly/3.6.10-candidates//build1/mac [ffxbld@surf mac]$ ls | grep -v xpi | wc -l 76 [ffxbld@surf linux-i686]$ cd /home/ftp/pub/firefox/nightly/3.6.10-candidates//build1/unsigned/win32 [ffxbld@surf win32]$ ls | grep -v xpi | wc -l 76 [ffxbld@surf linux-i686]$ cd /home/ftp/pub/firefox/nightly/3.6.10-candidates//build1/linux-i686 [ffxbld@surf linux-i686]$ ls | grep -v xpi | wc -l 76
(We need 76 builds per platform, per shipped-locales: http://hg.mozilla.org/releases/mozilla-1.9.2/file/FIREFOX_3_6_10_RELEASE/browser/locales/shipped-locales)
Return slaves
As the repacks finished I moved the following machines back to production-master01/03:
- Mac: bm-xserve06 and 07
- Win32: mw32-ix-slave06,12
- Linux: mv-moz2-linux-ix-slave03,04, linux-ix-slave17
I kept 2 Linux, 1 Mac, and 1 Windows machine for l10n verify / updates / update verify
XULRunner Builds
No problems
XULRunner Sign
Brought forward from 3.6.9 notes.
#!/bin/bash
set -x
set -e
PRODUCT=xulrunner
VERSION=1.9.2.10
BUILD=1
TAG=FIREFOX_3_6_10_RELEASE
REPO="releases/mozilla-1.9.2"
EMAIL=bhearsum@mozilla.com
KEYDIR=d:/2009-keys
cd ~/hg-tools
hg pull
hg up -r ${TAG}
mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
make setup PRODUCT=${PRODUCT} VERSION=${VERSION} \
BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
--exclude=*.txt --exclude=*-symbols.zip \
xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/ \
unsigned-build${BUILD}
rsync -av --exclude=unsigned unsigned-build${BUILD}/ \
signed-build${BUILD}/
rsync -av unsigned-build${BUILD}/unsigned/ \
signed-build${BUILD}/
make checksum-files create-sigs stage verify-sigs \
PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} \
REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
rmdir signed-build${BUILD}/contrib{,-localized}
rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
signed-build${BUILD}/ \
xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/
Partner Repacks
Halted because we don't use them.
Unittests
Did not trigger because this release on the new master and we forgot to add sendchanges for it. Triggered manually with:
# cltbld @ production-master.b.m.o buildbot sendchange --master production-master.build.mozilla.org:9011 --username sendchange-unittest --branch release-mozilla-1.9.2-linux-opt-unittest --revision 16b7b02798df http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1/linux-i686/en-US/firefox-3.6.10.tar.bz2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1/linux-i686/en-US/firefox-3.6.10.tests.zip buildbot sendchange --master production-master.build.mozilla.org:9011 --username sendchange-unittest --branch release-mozilla-1.9.2-win32-opt-unittest --revision 16b7b02798df http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1/unsigned/win32/en-US/firefox-3.6.10.zip http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1/unsigned/win32/en-US/firefox-3.6.10.tests.zip buildbot sendchange --master production-master.build.mozilla.org:9011 --username sendchange-unittest --branch release-mozilla-1.9.2-macosx-opt-unittest --revision 16b7b02798df "http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1/mac/en-US/Firefox 3.6.10.dmg" "http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1/mac/en-US/Firefox 3.6.10.tests.zip"
Had to kill the Linux ones the first time because they were delaying update generation.
Results
- Linux: PASS
- Win32: Conditional PASS
- test_playback.html timed out in mochitests - known intermittent orange bug 557432
- Mac: Conditional PASS
- browser_library_middleclick.js failed - known orange bug 595363
- 6 failures in https://bugzilla.mozilla.org/show_bug.cgi?id=534889 -- did not get these diagnosed but LegNeato said he wasn't worried about them based on the changes we took for this release
Sign
Removed en-US.xpi from stage.
#!/bin/bash
set -x
set -e
PRODUCT=firefox
VERSION=3.6.10
BUILD=1
TAG=FIREFOX_3_6_10_RELEASE
REPO="releases/mozilla-1.9.2"
EMAIL=bhearsum@mozilla.com
KEYDIR=d:/2009-keys
cd ~/hg-tools
hg pull
hg up -r tip # repo wasn't tagged when signing was run
mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
make PRODUCT=${PRODUCT} VERSION=${VERSION} \
BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} \
KEYDIR=${KEYDIR} USE_NEW=1
Reviewed stdout.
make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} \
BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}
Got "Signature Verification Succeeded" mail.
L10N Verify
- Moved moz2-darwin9-slave02, 06, and 07 back to pm:8011 to get l10n verify running
- No problems with verify results
- Gave slaves back to pm01
Updates
3.6* -> 3.6.10
Generation
No problems with automation.
Snippets for 3.6.8 and 3.6.9 need to be adjusted to have an extv of '3.6.8' and '3.6.9, respectively.
# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
rsync -a 20100914-Firefox-3.6.10/ 20100914-Firefox-3.6.10-extv-fixed/
rsync -a 20100914-Firefox-3.6.10-beta/ 20100914-Firefox-3.6.10-extv-fixed-beta/
rsync -a 20100914-Firefox-3.6.10-test/ 20100914-Firefox-3.6.10-extv-fixed-test/
find 20100914-Firefox-3.6.10-extv-fixed*/Firefox/3.6.8 -type f -exec sed -i -e 's/^extv=3\.6\.10/extv=3.6.8/' {} \;
find 20100914-Firefox-3.6.10-extv-fixed*/Firefox/3.6.9 -type f -exec sed -i -e 's/^extv=3\.6\.10/extv=3.6.9/' {} \;
Checks
grep -hr ^ext 20100914-Firefox-3.6.10-extv-fixed*/Firefox/3.6.8 | uniq extv=3.6.8 grep -hr ^ext 20100914-Firefox-3.6.10-extv-fixed*/Firefox/3.6.9 | uniq extv=3.6.9 grep -hr ^ext 20100914-Firefox-3.6.10-extv-fixed*/Firefox/3.6.7 | uniq extv=3.6.10
and push live
~/bin/pushsnip 20100914-Firefox-3.6.10-extv-fixed-test
Update Verify
All green, nothing hidden in the logs. Ran before modifying the snippets but extv isn't used.
3.0.19 -> 3.6.9 MU
First, landed patcher configuration update and update verify config updates.
Generation
Done on mv-moz2-linux-ix-slave17:
mkdir -p /builds/slave/3.0.19-3.6.10-major-update/snippets cd /builds/slave/3.0.19-3.6.10-major-update/snippets cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher cd patcher cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot # build tools ./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log # download mars ./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log # FIXME - patcher needs to see that the MARs that it thinks are partials # are there or else it will not attempt to generate patchinfo cd temp/firefox ln -s 3.6.10 3.0.19-3.6.10 cd ../.. # Create partial patches and snippets ./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
Quick Verification
cd /builds/slave/3.0.19-3.6.10-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.10/
# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'
# beta == release
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/aus2\.beta/aus2/; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/aus2/aus2.beta/; system("diff -r -u $_ $a");'
Push to AUS
cd /builds/slave/3.0.19-3.6.10-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.10 rsync -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100915-Firefox-3.0.19-3.6.10-MU-test rsync -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100915-Firefox-3.0.19-3.6.10-MU-beta rsync -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100915-Firefox-3.0.19-3.6.10-MU
Enable test snippets
~/bin/backupsnip 20100915-Firefox-3.0.19-3.6.10-MU-test ~/bin/pushsnip 20100915-Firefox-3.0.19-3.6.10-MU-test
Verify
On mv-moz2-linux-ix-slave17, mw32-ix-slave16, bm-xserve17
mkdir -p /builds/slave/verify/firefox-3019-3610-major/ cd /builds/slave/verify/firefox-3019-3610-major/ hg clone http://hg.mozilla.org/build/tools cd tools/release/updates platform=linux # or mac or win32 ./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log
Results
Checked the logs carefully and found the same differences documented for 3.0.19 -> 3.6's major update.
Update Bouncer
Used "force build" to start bouncer_submitter builder. No issues.
EU Ballot Build
Signed with: https://bugzilla.mozilla.org/show_bug.cgi?id=596597#c1.
Staged with:
# ffxbld@stage # get copy of signed tarball to /tmp/ mkdir /tmp/ffxbld cd /tmp/ffxbld tar xfv ../euballot-3610-signed.tar mv win32 win32-EUballot mv -v win32-EUballot /pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1/
Virus Scan
#ffxbld@stage cd /pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1 clamscan -r . ----------- SCAN SUMMARY ----------- Known viruses: 830883 Engine version: 0.96.1 Scanned directories: 661 Scanned files: 1528 Infected files: 152 Data scanned: 11139.62 MB Data read: 9146.89 MB (ratio 1.22:1) Time: 3869.757 sec (64 m 29 s) (arg: 4)
All win32 EXEs were reported to have "Trojan.Generic.Bredolab-2 FOUND", which also happened in 4.0b6 and was determined to be a false positive. Also ran it through VirusTotal, which showed it to be 100% clean. Based on those two things, working on the assumption that it's a false positive.
update: after the first scan was done the virus definitions on stage were updated. I rescanned the en-US installer and it reported clean.
Final Checks Before Push
Then do the normal checks
# ffxbld@stage cd /pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1/ find . ! -user ffxbld | grep -v 'contrib*' find . ! -group firefox | grep -v 'contrib*' find . -type f ! -perm 644 | grep -v 'unsigned*' find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*' | grep -v unsigned find . -maxdepth 1 -type d ! -perm 2775 -name 'contrib*'
No output from any of these checks (no contrib builds yet, no partner builds were generated).
Push to mirrors
After GO from release-drivers.
#ffxbld@stage rsync -av --exclude=*.log --exclude=*.txt --exclude=*unsigned* --exclude=*.crashreporter-symbols.zip --exclude=*.tests.tar.bz2* --exclude=*partner-repacks* /pub/mozilla.org/firefox/nightly/3.6.10-candidates/build1/ /pub/mozilla.org/firefox/releases/3.6.10/
#cltbld@stage vim /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude # replace 3.6.10 with 3.6.9
Final verification (sanity check releasetest snippets)
Push release/beta channel snippets
~/bin/backupsnip 20100914-Firefox-3.6.10-extv-fixed
Pushed the release channel snippets to get a head start, under the assumption that most users that could receive them would be throttled:
~/bin/pushsnip 20100914-Firefox-3.6.10-extv-fixed
But Nick pointed out that 3.6.9 isn't throttled, so I turned them off with:
# root@aus2-staging cd /opt/aus2/incoming chmod 0 3 cd 3/Firefox chmod 0 3.6.9
In order to check 3.5.x, changed this to only block 3.6.x
# root@aus2-staging cd /opt/aus2/incoming/3/Firefox ls | grep '3\.6.*' | xargs chmod 0 cd ../.. chmod 775 3
And later to check releasetest on 3.6.x, and block only release
# root@aus2-staging
cd /opt/aus2/incoming/3/Firefox
find 3.6* -maxdepth 4 -mindepth 4 -type d -name 'release' -print -exec chmod 0 {} \;
ls | grep '3\.6.*' | xargs chmod 775
Get go from driver, re-enable 3.6.x on release
# root@aus2-staging
cd /opt/aus2/incoming/3/Firefox
find 3.6* -maxdepth 4 -mindepth 4 -type d -name 'release' -print -exec chmod 775 {} \;
Modify the beta snippets to use bouncer too (saves on QA, load on ftp.m.o)
# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
rsync -a 20100914-Firefox-3.6.10-extv-fixed/ 20100914-Firefox-3.6.10-extv-bouncer-beta/
cd 20100914-Firefox-3.6.10-extv-bouncer-beta/
find . -maxdepth 5 -mindepth 5 -type d -exec mv -v {}/release {}/beta \;
~/bin/backupsnip 20100914-Firefox-3.6.10-extv-bouncer-beta
~/bin/pushsnip 20100914-Firefox-3.6.10-extv-bouncer-beta
Push throttling changes
Patch landed in cvs, deployment is bug 596849.
Release
Update symlinks
#ffxbld@stage cd /home/ftp/pub/firefox/releases rm latest-3.6 && ln -s 3.6.10 latest-3.6
Sent the following mail to metrics@mozilla.com:
Firefox,firefox,firefox,3.6.10,3.6 Firefox,firefox,firefox,3.6.11pre,3.6
Push EU Ballot build
Added bouncer product and location. Filed bug 596859 to have the browser choice website updated.
Push 3.0.19 -> 3.6.10 MU snippets (beta and release)
As cltbld@aus2-staging:
~/bin/backupsnip 20100915-3.0.19-3.6.10-MU ~/bin/pushsnip 20100915-3.0.19-3.6.10-MU
After QA signed off on the release channel:
~/bin/pushsnip 20100915-3.0.19-3.6.10-MU-beta
Push XULRunner to mirrors
# xrbld@stage rsync -av --exclude=*.txt --exclude=*unsigned* --exclude=*-symbols.zip \ /pub/mozilla.org/xulrunner/nightly/1.9.2.10-candidates/build1/ \ /pub/mozilla.org/xulrunner/releases/1.9.2.10/
Edited cltbld@stage:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude and added
- xulrunner/releases/1.9.2.9
Update XULRunner entries on wiki
Updated the links in these two wiki pages for the version change:
