Releases/Firefox 4.0b4/BuildNotes

Build Engineers

Signed-off Revision(s)

Build1: b51b190b9fcc
Build2: 9aa39b619a19
Build3: 9d6448b6a677

L10N changesets

using l10n dashboard I grabbed the l10n-changesets for the config update patch, then hit "Ship it!" Note: if there are pending sign-offs, ping Axel to make sure it's ok to grab the changesets

Build data

Build #	Type	Build ID	Build machine	Time to build
2	[Linux compressed]	20100805181633	mv-moz2-linux-ix-slave10	52 mins
	[Linux 64 compressed]	20100817135329	moz2-linux64-slave06	2 hrs, 7 mins
	[Mac OS X]	20100817134330	bm-xserve11	3 hrs, 28 mins
	[Mac OS X 64]	20100817134655	moz2-darwin10-slave05	1 hrs, 45 mins
	[Win32]	20100817145336	mw32-ix-slave03	2 hrs, 44 mins
3	[Linux compressed]	20100818120848	mv-moz2-linux-ix-slave10	59 mins
	[Linux 64 compressed]	20100818121643	moz2-linux64-slave06	1 hrs, 51 mins
	[Mac OS X]	20100818121614	bm-xserve11	3 hrs, 31 mins
	[Mac OS X 64]	20100818121922	moz2-darwin10-slave05	1 hrs, 48 mins
	[Win32]	20100818132640	mw32-ix-slave03	2 hrs, 48 mins

Bugs that we have hit on this release

~~bug 588546 - Unittests for all but win32 are not being triggered, packageTests step not run either~~

NOTE from armenzg: There might be more on the build notes but I have added the one that I have fixed for the next beta.

Build Notes

Build 1

Marked all builders for all releases on pm for a clobber -- https://build.mozilla.org/clobberer/ has a Release Clobber option if you are not connected to Build VPN

Moved the following slaves to pm:
- from pm03:
  - mv-moz2-linux-ix-slave10
  - mw32-ix-slave18
  - mw32-ix-slave15
- from pm01:
  - mv-moz2-linux-ix-slave02
  - mv-moz2-linux-ix-slave03
  - mv-moz2-linux-ix-slave04
  - mw32-ix-slave02
  - mw32-ix-slave03
  - bm-xserve06
  - bm-xserve11
  - bm-xserve12
  - bm-xserve16
  - moz2-linux64-slave04
  - moz2-linux64-slave06
  - moz2-darwin10-slave05
  - moz2-darwin10-slave06

Landed config patch
Reconfigured pm (delete *.pyc files)
- needed to delete some files on pm since it was out of space (deleted /tmp/tmp* and got 2.3 GB)
sendchange to get things running:

buildbot sendchange --username=lsblakk --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b4 build1" Go

Tag

mv-moz2-linux-ix-slave04

I had not reconfigured pm (all that fun with checkconfig messing up distracted me!) and so the first run of tag did 4.0b3 tagging instead
Beltzner had already given the request to cancel builds and wait for new additions to the 4.0b4 build2 so I re-did tagging (on the same slave) with force build after reconfiging and clobbering.
Re-did build1 tagging with the correct config to cut the relbranch.

After tagging completed we stopped for:

https://bugzilla.mozilla.org/show_bug.cgi?id=587922 (fix needed)
https://bugzilla.mozilla.org/show_bug.cgi?id=585877 (backout)

Build 2

Removed *pyc on pm moz2-master
Clobbered the release builders (NOTE: do not be connected to Build-VPN)
Used same l10n-changesets as build1 since Axel did not respond to ping
Landed configs
Reconfig & close m-c tree
Sendchange:

buildbot sendchange --username=lsblakk --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b4 build2" go

Tag

mv-moz2-linux-ix-slave04

tagged buildbot-configs, buildbotcustom, tools with FIREFOX_4_0b4_BUILD2, FIREFOX_4_0b4_RELEASE (and also did BUILD1 since that hadn't happened yet)

no problems

Source

mv-moz2-linux-ix-slave04

no problems

Build/Repack

No Problems on all platforms (linux,linux64,macosx,macosx64,win32)

Unittests

Sendchanges were not correctly formed in the automation (bug 585098 was checked in but pm buildbotcustom had not been updated) so I manually did sendchanges to get linux/linux64/macosx/macosx64 results:

Unittest:

buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-linux-opt-unittest --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/linux-i686/en-US/firefox-4.0b4.tar.bz2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/linux-i686/en-US/firefox-4.0b4.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-linux64-opt-unittest --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/linux-x86_64/en-US/firefox-4.0b4.tar.bz2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/linux-x86_64/en-US/firefox-4.0b4.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-macosx-opt-unittest --revision 9aa39b619a19 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/mac/en-US/Firefox 4.0 Beta 4.dmg' 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/mac/en-US/Firefox 4.0 Beta 4.tests.zip'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-macosx64-opt-unittest --revision 9aa39b619a19 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/mac64/en-US/Firefox 4.0 Beta 4.dmg' 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/mac64/en-US/Firefox 4.0 Beta 4.tests.zip'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-win32-opt-unittest --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/unsigned/win32/en-US/firefox-4.0b4.zip http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/unsigned/win32/en-US/firefox-4.0b4.tests.zip

Talos:

buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-linux-talos --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/linux-i686/en-US/firefox-4.0b4.tar.bz2
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-linux64-talos --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/linux-x86_64/en-US/firefox-4.0b4.tar.bz2
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-win32-talos --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/unsigned/win32/en-US/firefox-4.0b4.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-macosx-talos --revision 9aa39b619a19 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/mac/en-US/Firefox 4.0 Beta 4.dmg'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-macosx64-talos --revision 9aa39b619a19 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/mac64/en-US/Firefox 4.0 Beta 4.dmg'

Results

Linux/Linux64/Macosx64 builds did not make tests, upload tests, thus failed tests
Win32 had a green run on the tests run on pm, but had intermittent orange on the ones run on minis (two runs, since I triggered a second set manually)
Macosx ran orange on all but jsreftest and reftest

Signing

en-US.xpi removed from stage bug 485860:

#ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3
rm en-US.xpi

Using instructions.

L10N Verify

metadiff for win32_l10n had warnings - on many locales

source/bin/chrome/${locale}.jar and target/bin/chrome/${locale}.jar differ

Updates

mv-moz2-linux-ix-slave03

first run of updates blew up on downloading 75/363, I moved updates/builds dir aside and tried again and got past 75 on the downloading.

Build 3

missed the maxVer bump required for the Feedback Add-On
clobbered release builders
no l10n changes - checked with Axel in #release-drivers
reconfig pm (Note: updated buildbotcustom before this build, forgot to remove *.pyc will have to wait and see if that causes any problems)
notified #developers (changed topic even!) and closed tree for tagging
- Why do we close the tree for tagging? Our automation doesn't cope well with losing a push race. bug 508896
sendchange:

buildbot sendchange --username=lsblakk --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b4 build3" doit

Tag

mv-moz2-linux-ix-slave04

tagged buildbot-configs, buildbotcustom, tools with FIREFOX_4_0b4_BUILD3, FIREFOX_4_0b4_RELEASE

no problems

Source

No problems.

Build/Repack

No problems

Unittests

Filed bug 588546 because of a regression from bug 585098 where now only the unittestPlatforms specified in the release_config get the make packageTest step run. This results in only sendchanges/tests for win32 -- note in build2 also got test package for macosx, but see if that still happens now that buildbotcustom was updated for this run

Results

win32 tests run on pm had a timeout on xpcshell consistent with bug 580525

Signing

en-US.xpi removed from stage bug 485860:

#ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3
rm en-US.xpi

Using instructions.

L10N Verify

no problems Only in... for all platforms was mostly the same except some ja/ja-JP differences (we will be symlinking the ftp later for that locale)
win32 currently gives warnings on the jar diff step -- this appears to be new since bug 552668 and was not present in previous beta releases

source/bin/chrome/${locale}.jar and target/bin/chrome/${locale}.jar differ

- bug 589196 (new setup layout for win32) fixed it (see #Update_Verify for the details)

Updates

no problems

Update Verify

After some investigation bug 589196 was filed (Windows setup layout was changed in beta 4). After landing the patch the builder became green.
The output has the following expected diffs:

Only in source/bin/chrome: be.manifest
Only in source/bin/defaults/profile: prefs.js

- $locale.manifest became localized.manifest in b4 (bug 579178, bug 556644)
- prefs.js was removed in b4

Move slaves back to active masters

Returned all but 1 slave (kept mv-moz2-linux-ix-slave02) to pm01/03:

to pm03:
- mv-moz2-linux-ix-slave10
- mw32-ix-slave18
- mw32-ix-slave15
to pm01:
- mv-moz2-linux-ix-slave03
- mv-moz2-linux-ix-slave04
- mw32-ix-slave02
- mw32-ix-slave03
- bm-xserve06
- bm-xserve11
- bm-xserve12
- bm-xserve16
- moz2-linux64-slave04
- moz2-linux64-slave06
- moz2-darwin10-slave05
- moz2-darwin10-slave06

Final checks before push

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/
find . ! -user ffxbld # contib/...
find . ! -group firefox
find . -type f ! -perm 644
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*' # contib/...
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*' # contib/...
^^ Last line is WRONG, should be |find . -maxdepth 1 -type d ! -perm 2775 -name 'contrib*'|

Virus check

clamdscan -m /mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/4.0b4-candidates/build3/
/mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/4.0b4-candidates/build3: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 1056.363 sec (17 m 36 s)

Update Bouncer

Forced bouncer_submitter builder. Checked using web interface.

Push to mirrors

# ffbld@stage
rsync -av --exclude=*tests* --exclude=*crashreporter* --exclude=*.log \
  --exclude=*.txt --exclude=*unsigned* --exclude=*update-backup* \
  /pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/ \
  /pub/mozilla.org/firefox/releases/4.0b4

Add index.html to cover the files before release time:

# ffbld@stage
cd /pub/mozilla.org/firefox/releases/4.0b4
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=456169 | sed 's/Beta 1/Beta 4/g' > index.html
for dir in `find . -mindepth 1 -type d`; do cp -pv index.html $dir/; done

Final verify

went green on first run - sent remaining slave (mv-moz2-linux-ix-slave02) back to pm01
sent email when mirrors were absorbed for testing at 8:07am PDT
sent email when mirrors were absorbed for release at 9:17am PDT (more than 30K uptake)

Push snippets to beta channel

Upon GO from release-drivers

# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip 20100817-Firefox-4.0b4
~/bin/pushsnip   20100817-Firefox-4.0b4

After investigating why the beta updates did not apply cleanly (and throttling updates temporarily it turned out that I had pushed build2 snippets live

# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
~/bin/pushsnip   20100818-Firefox-4.0b4

Pinged QA to re-test
- QA bug: updates do not apply cleanly bug 590235
- ServerOps blocker bug to throttle bug 590227
- ServerOps blocker bug to unthrottle bug 590250

Remove index.html files

Once we are fully announced (by the driver) we can remove the index.html files:

# ffxbld@stage
cd /pub/mozilla.org/firefox/releases/4.0b4/
find . -name index.html
find . -name index.html | xargs rm -v

Release

Emailed the following to metrics@m.c

Firefox,firefox,firefox,4.0b4,4.0
Firefox,firefox,firefox,4.0b5pre,4.0

Build #	Tag	Changeset
1	GECKO20b4_20100817_RELBRANCH	b51b190b9fcc
1	FIREFOX_4_0b4_BUILD1, FIREFOX_4_0b4_RELEASE	cb39cf6342e5
2	FIREFOX_4_0b4_BUILD2, FIREFOX_4_0b4_RELEASE	9aa39b619a19
3	FIREFOX_4_0b4_BUILD3, FIREFOX_4_0b4_RELEASE	9d6448b6a677

Releases/Firefox 4.0b4/BuildNotes

Contents

Build Engineers

Signed-off Revision(s)

L10N changesets

Tags

Build data

Bugs that we have hit on this release

Build Notes

Build 1

Tag

Build 2

Tag

Source

Build/Repack

Unittests

Results

Signing

L10N Verify

Updates

Build 3

Tag

Source

Build/Repack

Unittests

Results

Signing

L10N Verify

Updates

Update Verify

Move slaves back to active masters

Final checks before push

Virus check

Update Bouncer

Push to mirrors

Final verify

Push snippets to beta channel

Remove index.html files

Release

Navigation menu

Search