Releases/Firefox 3.5.2/BuildNotes
Contents
- 1 Build Engineers
- 2 Signed-off Revision(s)
- 3 Tags
- 4 Build data
- 5 Notes
- 5.1 Build 1
- 5.1.1 Tag
- 5.1.2 Source
- 5.1.3 Build/Repack
- 5.1.4 Sign
- 5.1.5 L10nVerify
- 5.1.6 Generate updates and push betatest snippets
- 5.1.7 Update verify
- 5.1.8 Publish updates to beta channel
- 5.1.9 Update Bouncer
- 5.1.10 Final checks before push
- 5.1.11 Push to mirrors
- 5.1.12 Final Verification
- 5.1.13 Push updates to release channel
- 5.1.14 Release
- 5.1.15 XULRunner build
- 5.1 Build 1
Build Engineers
nthomas, bhearsum
Signed-off Revision(s)
Build1: 7a9fe7dfac8a
Tags
On releases/mozilla-1.9.1:
Build # | Tag | Changeset |
1 | GECKO1912_20090729_RELBRANCH | 7a9fe7dfac8a |
FIREFOX_3_5_2_BUILD1, FIREFOX_3_5_2_RELEASE | Was:b8dbd5ab1647 Became:d57f05bcdbe1 Then:a6308d41af58 |
Build data
Type | Build ID | Build machine |
[Windows installer/zip] | ||
[Mac compressed] | ||
[Linux compressed] |
Notes
Build 1
- Cleaned up previous releases' build dirs
- l10n-changesets updated in bug 505129
- Land version bump, reconfig production-master
- Kick-off automation:
buildbot sendchange --username=nthomas --master=localhost:9010 --branch=releases/mozilla-1.9.1 -m "Firefox 3.5.2 build1" goforit
Tag
No problems with tag (see below for where it got moved)
Source
No problems
Build/Repack
Had to remove en-US.xpi manually from stage FIXME No problems generating en-US builds, but linux locales started failing in the make_installers_locale step after generating tar.bz2 and complete.mar:
/tools/python/bin/python ../../tools/update-packaging/generatesnippet.py \ --mar-path=/builds/moz2_slave/linux_repack/build/mozilla-1.9.1/dist/update \ --application-ini-file=/builds/moz2_slave/linux_repack/build/mozilla-1.9.1/dist/l10n-stage/firefox/application.ini \ --locale=or \ --product=Firefox \ --download-base-URL= \ --verbose /tools/python/bin/python: can't open file '../../tools/update-packaging/generatesnippet.py': [Errno 2] No such file or directory
This is fallout from a bad merge in bug 489313, as generatesnippet.py did not land on mozilla-1.9.1. Mac also hit it because it's not platform-dependent.
Recovery
- did a no-op reconfig on master to prevent win32 repacks from starting
- Fix was landed on the relbranch and default, and the two tags moved up to d57f05bcdbe1
- automation patched to disable tag and en-US builds
- cleaned up mac and linux slaves of repack dirs and source
- submitted another sendchange
Mac repacks were broken because of bug 489313. Root cause hasn't been determined yet, see below for fixup.
mac repack rerun
- Backed out bug 489313 on the release branch and moved the _BUILD1 and _RELEASE tags up.
- cleaned up the macosx_repack slave
- Deleted all l10n builds in .../build1/mac and ../build1/update/mac on stage.
- Landed this patch, updated the master and reconfiged
- Did another sendchange
no problems with the re-run. Detached PGP sigs were regenerated when resigning windows repacks.
win32 repack rerun
Don't need to land anything in mozilla-1.9.1 because the backout we did for mac will suffice
- cleaned up the win32_repack slaves
- Moved all l10n builds in .../build1/unsigned/win32, .../build1/unsigned/update/win32, .../build1/update/win32, .../build1/win32 to ~ffxbld/3.5.2-busted/
- Landed this patch, updated the master and reconfiged
- did another sendchange
- temporarily enabled some extra slaves (up to slave16) for win32 repacks to speed things up
- this ended up backfiring a little after removing them from the l10n slaves list before they finished building - it cause the builds running on slave11 and slave14 to hang, and they had to be kicked through manually
Sign
- Kicked off new signing on cm-keymaster01 and old style on keymaster at the same time
- new signing bailed out at verify-asc because SUMS files were not signed, but verify-asc thought they should be (FIXME). we were trying to push out signed *SUMS files this time. to save time but since we've never done it in the past, just pushed things along with unsigned *SUMS files by running the 'fake-upload' and then 'postsign' targets
signing # 2
Had to resign after win32 l10n builds were regenerated. sign-release.py was called directly to add -p argument, then worked through the following makefile targets. asc files for linux and mac were replaced.
L10nVerify
After redoing macosx locales: PASS. All locales with real changes have an updated l10n-changesets revision (got that list by doing 'hg diff -r FIREFOX_3_5_1_RELEASE -r FIREFOX_3_5_2_RELEASE mozilla2/l10n-changesets)
After redoing win32 locales: looks fine, in particular no win32 specific problems
Generate updates and push betatest snippets
- Update generation had 'WARNING' messages about not being able to find application.ini on Mac. This comes from get_buildid in make_incremental_updates.py, but we never actually use the output of that function which makes this a superfluous message. Noticed it in the 3.0.12 logs, too. bug 507405
- updates were stopped midstream after the mac problem was discovered and re-kicked with 'force build' after mac builds were fixed. this caused the 'diff patcher config' step to turn red because it expected diff output, but the config had already been bumped.
- no real problems then
After fixing win32 locales:
do these steps to generate updats # cltbld@moz2-linux-slave06 cd /builds/slave/updates/build rm -rv temp/firefox/3.5.2/ftp/firefox-3.5.2.*.win32.complete.mar rsync -a temp/firefox/3.5.1-3.5.2/ temp/firefox/3.5.1-3.5.2-bak/ rm -rf temp/firefox/3.5.1-3.5.2/ftp/firefox/nightly/3.5.2-candidates/build1/update/win32/{af,ar,as,be,bg,bn-BD,bn-IN,ca,cs,cy,da,de,el,en-GB,eo,es-AR,es-CL,es-ES,es-MX,et,eu,fa,fi,fr,fy-NL,ga-IE,gl,gu-IN,he,hi-IN,hr,hu,id,is,it,ja,ka,kk,kn,ko,ku,lt,lv,mk,ml,mn,mr,nb-NO,nl,nn-NO,oc,or,pa-IN,pl,pt-BR,pt-PT,rm,ro,ru,si,sk,sl,sq,sr,sv-SE,ta,ta-LK,te,th,tr,uk,vi,zh-CN,zh-TW} rm -rf temp/firefox/3.5.1-3.5.2/aus2* perl patcher2.pl --download --app=firefox --brand=Firefox --config=patcher-configs/moz191-branch-patcher2.cfg perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/moz191-branch-patcher2.cfg find temp/firefox/3.5.1-3.5.2/ftp/firefox/nightly/3.5.2-candidates/build1 -type f -exec chmod 644 {} \; find temp/firefox/3.5.1-3.5.2/ftp/firefox/nightly/3.5.2-candidates/build1 -type d -exec chmod 755 {} \; cd temp/firefox/ # for each of aus2, aus2.test, aus2.beta: verify that only WINNT have changed but not en-US, # of differences is the # of partials rsync -av -e "ssh -oIdentityFile=~/.ssh/ffxbld_dsa" --exclude=*complete.mar 3.5.1-3.5.2/ftp/firefox/nightly/3.5.2-candidates/build1/update ffxbld@stage-old.mozilla.org:/pub/mozilla.org/firefox/nightly/3.5.2-candidates/build1/ rsync -av 3.5.1-3.5.2/aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090730-Firefox-3.5.2-fixedWin32 rsync -av 3.5.1-3.5.2/aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090730-Firefox-3.5.2-fixedWin32-test rsync -av 3.5.1-3.5.2/aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090730-Firefox-3.5.2-fixedWin32-beta ssh -l cltbld aus2-staging.mozilla.org ~/bin/pushsnip 20090730-Firefox-3.5.2-fixedWin32-test
Update verify
- win32 update verify logs showed that the helper.exe in the installers, and the helper.exe in the MARs was different. this is also fallout from bug 489313, just like the mac problems. win32 OK otherwise
- linux - full PASS
- mac - full PASS
Publish updates to beta channel
# login with LDAP account onto aus2-staging, then do # sudo su - cltbld cd /opt/aus2/snippets/staging time ~/bin/backupsnip 20090730-Firefox-3.5.2-fixedWin32-beta real 47m29.661s user 0m46.668s sys 1m8.102s time ~/bin/pushsnip 20090730-Firefox-3.5.2-fixedWin32-beta real 2m28.442s user 0m0.382s sys 0m20.070s
Update Bouncer
done
Final checks before push
Each of the find calls should return no output.
cd /pub/mozilla.org/firefox/nightly/3.5.2-candidates/build1/ find . ! -user ffxbld find . ! -group firefox find . -type f ! -perm 644 find . -type d -mindepth 1 ! -perm 755 ! -name 'contrib*' find . -type d -maxdepth 1 ! -perm 2775 -name 'contrib*'
No Solaris builds at this point, so no output from any of these commands.
Push to mirrors
# ffxbld@stage rsync -av --exclude=*.log --exclude=*.txt --exclude=*unsigned* /pub/mozilla.org/firefox/nightly/3.5.2-candidates/build1/ /pub/mozilla.org/firefox/releases/3.5.2/
# cltbld@stage vim /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude # replace 3.5.1 with 3.5.2
Final Verification
- kick off 'final_verification' builder on production-master after we have some uptake. if it goes green everything is OK, if red look at the log for 404 or 503 errors.
- got nine FAIL's trying to query mozilla-mirror.3347.voxcdn.com and victoria.acc.umu.se, now working fine
Push updates to release channel
# cltbld @ aus2-staging cd /opt/aus2/snippets/staging screen # Skipped this time, because the earlier backupsnip was just a few hours ago time ~/bin/backupsnip 20090730-Firefox-3.5.2-fixedWin32 real 50m56.618s user 0m46.908s sys 1m0.209s time ~/bin/pushsnip 20090730-Firefox-3.5.2-fixedWin32 real 1m6.163s user 0m0.154s sys 0m5.773s
Release
# login as cltbld to any of the production build machines. From there, do: # ssh -l ffxbld -i ~/.ssh/ffxbld_dsa stage.m.o cd /home/ftp/pub/firefox/releases rm latest-3.5 && ln -s 3.5.2 latest-3.5
XULRunner build
Tracked separately in bug 505701.
Create the builds by going to the buildbot waterfall for mozilla-1.9.1 and forcing builds for linux (not x64), mac and windows xulrunner. Set the branch to 'releases/mozilla-1.9.1' and revision to 'FIREFOX_3_5_1_RELEASE'.
When they finish, as xrbld@stage move the dir from /pub/mozilla.org/xulrunner/nightly/2009/07/2009-07-xx-xx-mozilla-1.9.1/ to ..../nightly/1.9.1.1-candidates/build1. Remove the broken symlink at xulrunner/nightly/2009-07-xx-xx-mozilla-1.9.1. Contact mfinkle and dtownsend for smoketest QA.
Publishing is done in a similar (but not identical!) way as 3.0.13:
# cltbld@stage VERSION=1.9.1.2 FX_VERSION=3.5.2 BUILD=1 mkdir -p ~/xulrunner-${VERSION}/batch1/stage-unsigned/{runtimes,source,sdk} cd ~/xulrunner-${VERSION}/batch1/stage-unsigned rsync -av --exclude=*sdk* --exclude=*info* \ /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ runtimes/ rsync -av --include=*sdk* --exclude=* \ /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ sdk/ cp -pv /pub/mozilla.org/firefox/releases/${FX_VERSION}/source/firefox-${FX_VERSION}-source.tar.bz2 \ source/xulrunner-${VERSION}-source.tar.bz2 cd ~/xulrunner-${VERSION}/batch1 rsync -av stage-unsigned/ stage-signed/
Then create detached signatures per usual process (with PRODUCT=xulrunner, VERSION=1.9.0.13, don't need BUILD or TAG), skip the win32 signing section, do pgp, verify and upload. Back on stage:
cd ~/xulrunner-${VERSION}/batch1/stage-signed cp /pub/mozilla.org/firefox/releases/${FX_VERSION}/KEY . ~/bin/checksum-files . chown -R cltbld:xulrunner . find . -type f -exec chmod -v 644 {} \; find . -type d -exec chmod -v 755 {} \; cd ~/xulrunner-${VERSION}/batch1 rsync -nav stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/
If all is fine then
rsync -av stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/
Update the links in these two wiki pages for the version change