Releases/Firefox 3.0.14/BuildNotes

Build Engineer(s)

lsblakk

Tracking release bug

Bonsai queries

cvsroot

l10n

Notes

Build 1

On fx-win32-1.9-slave2:
- Deleted /e/xr19rel/WINNT_5.2_Depend
- Deleted /e/fx19l10nrel/WINNT_5.2_Depend
- Deleted /e/fx19rel/WINNT_5.2_Depend

On fx-linux-1.9-slave2:
- Deleted /builds/verify/firefox-3.0.13
- Deleted /builds/verify/firefox-3012-351-major
- Deleted /builds/updates/firefox-3.0.13
- Deleted /builds/source/firefox-3.0.13
- Deleted /builds/tags/FIREFOX_3_0_9_BUILD*
- Deleted /builds/data/cltbld/firefox-3.0.13

On fx-mac-1.9-slave2:
- Deleted /builds/verify/irefox-20020-3010-major
- Deleted builds/verify/firefox-3012-351-major/

Space free on slaves:
- fx-mac-1.9-slave2: 21G free on /
- fx-win32-1.9-slave2: 16.4G free on D, 14.1G free on E
- fx-linux-1.9-slave2: 20G free on /builds

Kicked off automation:

 buildbot sendchange --username=lsblakk --master=localhost:9989 -m"Firefox 3.0.14build1 release" go

Tag

No problems

Source

No problems

Build & Repack

Linux - no problems
Mac - no problems
Win32 - no problems

Sign

Signing doc
no problems

L10nVerify

no problems - no l10n changes

Generate Updates

We ran into a problem on the very first Mac update creation - make_incremental_updates.py didn't have file paths in quotations for the mbsdiff call and so because there was a change to Contents/MacOS/plugins/Default Plugin the attempt to do a patch failed due to spaces in the shell command call.

Filed bug 512477 to fix this issue, clobbered builds/updates/firefox-3.0.14 on fx-linux-1.9-slave2, commented out the PatcherConfig step in production-1.9-master's master.cfg, reconfig'd and forced build on update again.

Publish Updates to Test Channels (betatest & releasetest)

This is now automated in the Update step (see logs for push)

Update Verify

linux - full PASS
win32 - .chk files differ, as always, otherwise find.
mac - full PASS

Stage

No problems

XULRunner

No problems

At this point the release process was interrupted by bug 510583

Build 2

On fx-win32-1.9-slave2:
- Deleted /e/xr19rel/WINNT_5.2_Depend
- Deleted /e/fx19l10nrel/WINNT_5.2_Depend
- Deleted /e/fx19rel/WINNT_5.2_Depend

On fx-linux-1.9-slave2:
- Deleted /builds/verify/firefox-3013-352-major
- Deleted /builds/verify/firefox-3.0.13
- Deleted /builds/verify/firefox-3012-351-major
- Deleted /builds/verify/firefox-3.0.14
- Deleted /builds/updates/firefox-3.0.14
- Deleted /builds/source/firefox-3.0.14
- Deleted /builds/tags/FIREFOX_3_0_10 & FIREFOX_3_0_11*
- Deleted /builds/data/cltbld/firefox-3.0.14
- Deleted /builds/tinderbox/Xr-Mozilla

On fx-mac-1.9-slave2:
- Deleted /builds/verify/*

Space free on slaves:
- fx-mac-1.9-slave2: 20G free on /
- fx-win32-1.9-slave2: 16.4G free on D, 14.1G free on E
- fx-linux-1.9-slave2: 25G free on /builds

Kicked off automation

 buildbot sendchange --username=lsblakk --master=localhost:9989 -m"Firefox 3.0.14build2 release" go

Tag

No problems

Source

No problems

Build & Repack

Linux - had to do a couple of force builds because on the first run through my attempt to recreate the tinderbox dir in fx-linux-1.9-slave2 (after accidentally wiping it for Fx-Mozilla1.9-Release) had bad symlinks, then a second force because of having to comment out the TinderConfig step
No problems on Win32 or Mac

Sign

Signing doc
NB: Doc change - Made the old signing steps more obvious so as to avoid trying to do them.
no problems

L10nVerify

No l10n changes at all

Generate Updates

No problems with update generation

Publish Updates to Test Channels (betatest & releasetest)

No problems

Update Verify

linux - full PASS
win32 - .chk files differ, as always, otherwise fine.
mac - full PASS

Stage

No problems

XULRunner

No problems

3.0.14 -> 3.5.x Major Update Refresh

version bumps for patcher config and release/update

Snippet Generation

mkdir -p /builds/3.0.14-3.5.3-major-update/snippets
cd /builds/3.0.14-3.5.3-major-update/snippets

cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:ffxbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.5.3 3.0.14-3.5.3
cd ../..

# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Quick Verify

Check that releasetest = beta = release.

cd temp/firefox/3.0.14-3.5.3
# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

# beta == release
find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; system("diff -r -u $_ $a");'

Push to AUS

cd /builds/3.0.14-3.5.3-major-update/snippets/patcher/temp/firefox/3.0.14-3.5.3
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090831-Firefox-3.0.14-3.5.3-MU-test
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090831-Firefox-3.0.14-3.5.3-MU

Enable test snippets

ssh -i ~/.ssh/aus2 cltbld@aus2-staging
cd /opt/aus2/snippets/staging

~/bin/backupsnip 20090831-Firefox-3.0.14-3.5.3-MU-test
~/bin/pushsnip   20090831-Firefox-3.0.14-3.5.3-MU-test

Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-3014-353-major/
cd /builds/verify/firefox-3014-353-major/
hg clone http://hg.mozilla.org/build/tools

# get patch to trawl "Only in" directories
cd tools/release/common
# on mac & linux
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=367544 | patch -p3
# on win32
wget --no-check-certificate -O patch https://bugzilla.mozilla.org/attachment.cgi?id=367544§
patch -p3 < patch

# everyone ...
cd ../updates
platform=linux   # or mac or win32
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log

Check logs for results - whatever is after "succeeded", known issues are here

results everything the same as 3.0.13 -> 3.5.3 MU. aka, everything is OK

Push updates to beta channel

About an hour before the official go-to-beta time do backupsnip

  # cltbld@aus2-staging.m.o
  # make sure scripts are up to date
  cd bin
  cvs update
  cd /opt/aus2/snippets/staging/
  ~/bin/backupsnip 20090827-Firefox-3.0.14-beta

Then on official "go" do pushsnip

  ~/bin/pushsnip 20090827-Firefox-3.0.14-beta

Sign Installers

A day or two before the scheduled release day

Done manually using these installer-signing-instructions here. NB: Doc location change, content unmodified.

On Stage complete stage-merged:

# on stage
cd /data/cltbld/firefox-3.0.14/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/

Create MD5 and SHA1 checksum files

# on stage
cd /data/cltbld/firefox-3.0.14/stage-merged/
~/bin/checksum-files .

Fix permissions & ownership (on the two SUM files, and the detached sigs)

chown -R cltbld:firefox .
chmod 644 *SUMS

Update Bouncer

Done.

Push to mirrors

Make sure that you use ffxbld for 'push to mirrors' , rather than cltbld (this is a change, but a desired one).

push the stage-merged directory to the releases area:

# on stage
rsync -av /data/cltbld/firefox-3.0.14/stage-merged/ /home/ftp/pub/firefox/releases/3.0.14/

edit the exclude file (as cltbld) /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to add the new release (3.0.14) and remove the previous release (3.0.13).

Final Verification

hg clone http://hg.mozilla.org/build/tools
cd tools/release/updates
cat moz19-firefox-*.cfg | grep -v major | sed 's/betatest/releasetest/' > update.cfg
./verify.sh -t update.cfg 2>&1 | tee quickVerify.log

Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):

grep HTTP quickVerify.log | grep -v 200 | grep -v 302

- No Problems

Before pushing final updates, verify that "release" and "releasetest" channel match:

# on aus2-staging
cd /opt/aus2/snippets/staging/20090827-Firefox-3.0.14
find -type d -iregex '.*release.*' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20090827-Firefox-3.0.14-test/$a");'

Publish Updates to Release Channel

Start doing backupsnip a few hours at least before the release time

# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
time ~/bin/backupsnip 20090827-Firefox-3.0.14
real	58m7.854s
user	0m48.085s
sys	1m5.146s

Wait for "go" to do pushupsnip for Firefox 3.0.14

time ~/bin/pushsnip 20090827-Firefox-3.0.14
real	3m7.936s
user	0m0.398s
sys	0m10.728s

Then again, waiting for go on MU

time ~/bin/pushsnip 20090821-Firefox-3.0.14-3.5.3-MU
real	0m11.882s
user	0m0.038s
sys	0m1.098s

Release

# ffxbld@stage
cd /home/ftp/pub/firefox/releases
rm latest-3.0 && ln -s 3.0.14 latest-3.0

Release matching XULRunner build

XULRunner builds are produced by the automation, this is to groom them and push to the mirrors.

# cltbld@stage
VERSION=1.9.0.14
FX_VERSION=3.0.14
BUILD=2

mkdir -p ~/xulrunner-${VERSION}/batch1/stage-unsigned/{runtimes,source,sdk}
cd ~/xulrunner-${VERSION}/batch1/stage-unsigned
rsync -av --exclude=*sdk* --exclude=*info* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ runtimes/
rsync -av --include=*sdk* --exclude=* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ sdk/
cp -pv ~/firefox-${FX_VERSION}/stage-merged/source/firefox-${FX_VERSION}-source.tar.bz2 \
  source/xulrunner-${VERSION}-source.tar.bz2
cd ~/xulrunner-${VERSION}/batch1
rsync -av stage-unsigned/ stage-signed/

Then create detached signatures per usual process (with PRODUCT=xulrunner, VERSION=1.9.0.13, don't need BUILD or TAG), skip the win32 signing section, do pgp, verify and upload. Back on stage:

cd ~/xulrunner-${VERSION}/batch1/stage-signed
cp ~/firefox-${FX_VERSION}/stage-merged/KEY .
~/bin/checksum-files .
chown -R cltbld:xulrunner .
find . -type f -exec chmod -v 644 {} \;
find . -type d -exec chmod -v 755 {} \;
cd ~/xulrunner-${VERSION}/batch1
rsync -nav stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

If all is fine then

rsync -av stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

(Mirror updating can take a bit, 10-15 mins this time)

Ping mfinkle or Mossop to update the links in these two wiki pages for the version change

Module	Branch	Tag	Pull date
cvsroot/mozilla	HEAD	GECKO190_20090824_RELBRANCH FIREFOX_3_0_14_BUILD1 FIREFOX_3_0_14_RELEASE
l10n/l10n	HEAD	GECKO190_20090824_RELBRANCH FIREFOX_3_0_14_BUILD1 FIREFOX_3_0_14_RELEASE

Module	Branch	Tag	Pull date
cvsroot/mozilla	HEAD	GECKO190_20090824_RELBRANCH FIREFOX_3_0_14_BUILD2 FIREFOX_3_0_14_RELEASE
l10n/l10n	HEAD	GECKO190_20090824_RELBRANCH FIREFOX_3_0_14_BUILD2 FIREFOX_3_0_14_RELEASE

Releases/Firefox 3.0.14/BuildNotes

Contents

Build Engineer(s)

Bonsai queries

Tags

Notes

Build 1

Tag

Source

Build & Repack

Sign

L10nVerify

Generate Updates

Publish Updates to Test Channels (betatest & releasetest)

Update Verify

Stage

XULRunner

Build 2

Tag

Source

Build & Repack

Sign

L10nVerify

Generate Updates

Publish Updates to Test Channels (betatest & releasetest)

Update Verify

Stage

XULRunner

3.0.14 -> 3.5.x Major Update Refresh

Snippet Generation

Quick Verify

Push to AUS

Enable test snippets

Update verify

Push updates to beta channel

Sign Installers

Update Bouncer

Push to mirrors

Final Verification

Publish Updates to Release Channel

Release

Release matching XULRunner build

Navigation menu

Search