Releases/Firefox 3.0.19/BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineer(s)

lsblakk

Tracking release bug

Bonsai queries

Tags

Build 1:

Module Branch Tag Pull date
cvsroot/mozilla HEAD GECKO190_20100313_RELBRANCH FIREFOX_3_0_19_BUILD1 FIREFOX_3_0_19_RELEASE 2010-03-10 13:13 PST
l10n/l10n HEAD GECKO190_20100313_RELBRANCH FIREFOX_3_0_19_BUILD1 FIREFOX_3_0_19_RELEASE 2009-09-22 05:34 PDT

Notes

Build 1:

  • on fx-linux-1.9-slave2:
    • rm -rf /builds/verify/firefox-3* /builds/source/firefox-3.0.18 /builds/data/cltbld/firefox-3.0.18 /builds/tinderbox/Fx-Mozilla1.9-l10n-Release/Linux_2.6.18-53.1.13.el5_Depend/ /builds/tinderbox/Fx-Mozilla1.9-Release/Linux_2.6.18-53.1.13.el5_Depend/ /builds/tags/* /builds/3.0.18-3.6-major-update/ /builds/updates/firefox-3.0.18
  • on fx-mac-1.9-slave2:
    • rm -rf /builds/verify/firefox-*
  • on fx-win32-1.9-slave2 deleted:
    • /e/xr19rel/
    • /e/fx19l10nrel/
    • /e/fx19rel/
    • Note: this was incorrect, should have deleted WINNT_5.2_Depend dirs in each, so the win32 builds failed the first time - had to recreate dirs
  • Available space on slaves:
    • fx-linux-1.9-slave2 36 GB on /builds
    • fx-mac-1.9-slave2 25 GB on /
    • fx-win32-1.9-slave2 13.8 GB on D: / 14.1 GB on E:
  • landed automation config and tagged with RELEASE_AUTOMATION_M15_1
  • tagged hg:build/tools with FIREFOX_3_0_19_RELEASE
  • kicked off automation:
buildbot sendchange --username=lsblakk --master=localhost:9989 -m"Firefox 3.0.19build1 release" RunForestRun!

Tag

No problems

Source

No problems

Build & Repack

  • Linux & Mac were fine. Because of my over-enthusiastic removal of directories when making space on the win32 slave, the win32 build failed. Nthomas restore the directory structure and the win32 build was re-started and went through without a hitch.

Sign

No problems

L10n Verify

Kicked off manually because win32 build fail killed the automation. No problems.

Generate Updates

Kicked off manually because win32 build fail killed the automation. No problems.

Publish Updates to Test Channels (betatest & releasetest)

No problems

Update Verify

All three platforms failed after 5+ hours of running update_verify. There were known network issues yesterday as per bug 552481. Just forced rebuild on all three to see if it works today.

Second run of update_verify failed with same results. Nthomas is going to help look into what's happening here - suspects it might have to do with apache changes to staging-old.

Removed /builds/logs/update_verify.log on slaves and kicked off update_verify again.

no problems aside from the normal .chk file differences. I used

curl http://production-1.9-master.build.mozilla.org:8810/builders/win32_update_verify/builds/22/steps/shell_8/logs/stdio 2>/dev/null|grep differ

to ensure that the only files that were no the same were softokn3.chk and freebl3.chk

Stage

no problems

XULRunner builds

No longer required (bug 544678)

3.0.19 -> 3.6 Major Update Refresh

Snippet Generation

on fx-linux-1.9-slave2:

mkdir -p /builds/3.0.19-3.6.2-major-update/snippets
cd /builds/3.0.19-3.6.2-major-update/snippets

cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.6.2 3.0.19-3.6.2
cd ../..

# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
Quick Verify

Check that releasetest = beta = release

cd temp/firefox/3.0.19-3.6.2

# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

# beta == release
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/aus2\.beta/aus2/; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/aus2/aus2.beta/; system("diff -r -u $_ $a");'
Push to AUS
cd /builds/3.0.19-3.6.2-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.2
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100317-Firefox-3.0.19-3.6.2-MU-test
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100317-Firefox-3.0.19-3.6.2-MU-beta
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/      cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100317-Firefox-3.0.19-3.6.2-MU
Enable test snippets
ssh -i ~/.ssh/aus2 cltbld@aus2-staging.mozilla.org

~/bin/backupsnip 20100317-Firefox-3.0.19-3.6.2-MU-test
~/bin/pushsnip   20100317-Firefox-3.0.19-3.6.2-MU-test
Update Verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

 mkdir -p /builds/verify/firefox-3019-362-major/
 cd /builds/verify/firefox-3019-362-major/
 hg clone http://hg.mozilla.org/build/tools
 
 cd tools/release/common
 cd ../updates
 platform=linux   # or mac or win32
 ./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log
Results

Linux

diff -r source/firefox/defaults/pref/channel-prefs.js target/firefox/defaults/pref/channel-prefs.js

Harmless conmment change

Locale-specific: leftover searchplugins

  • fr
Only in source/firefox/searchplugins: MediaDICO-fr.xml
  • hu
Only in source/firefox/searchplugins: amazondotcom.xml
Only in source/firefox/searchplugins: answers.xml
Only in source/firefox/searchplugins: eBay.xml
Only in source/firefox/searchplugins: yahoo.xml
  • it
Only in source/firefox/searchplugins: demauro.xml
  • sl
Only in source/firefox/searchplugins: iskalnik-gov-si.xml
Only in source/firefox/searchplugins: najdi-si.xml
Only in source/firefox/searchplugins: yahoo.xml
  • uk
Only in source/firefox/searchplugins: price-ua.xml

Locale-specific: no locale for 3.6.2

  • mn
FAIL: no partial update found for https://aus2.mozilla.org/update/1/Firefox/3.0.19/2010031218/Linux_x86-gcc3/mn/betatest/update.xml?force=1
FAIL: download_mars returned non-zero exit code: 1

Mac

diff -r source/Firefox.app/Contents/MacOS/defaults/pref/channel-prefs.js target/Firefox.app/Contents/MacOS/defaults/pref/channel-prefs.js
1c1
< //@line 2 "/builds/tinderbox/Fx-Mozilla1.9-Release/Darwin_8.8.4_Depend/mozilla/browser/app/profile/channel-prefs.js"
---
> //@line 2 "/builds/slave/macosx_build/build/browser/app/profile/channel-prefs.js"
Only in source/Firefox.app/Contents/MacOS/plugins: Default Plugin.plugin
Only in source/Firefox.app/Contents/MacOS/res/cursors: CVS
Only in source/Firefox.app/Contents: Plug-Ins
Contents of source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin dir only in source or target
1809783973    0 drwxr-xr-x    4 cltbld   cltbld        136 Mar 18 09:32 source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin/Contents
1809783974    0 drwxr-xr-x    2 cltbld   cltbld         68 Mar 18 09:32 source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin/Contents/MacOS
1809783975    0 drwxr-xr-x    3 cltbld   cltbld        102 Mar 18 09:32 source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin/Contents/Resources
1809783977    0 drwxr-xr-x    2 cltbld   cltbld         68 Mar 18 09:32 source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin/Contents/Resources/English.lproj
Contents of source/Firefox.app/Contents/MacOS/res/cursors/CVS dir only in source or target
Contents of source/Firefox.app/Contents/Plug-Ins dir only in source or target
1809784006    0 drwxr-xr-x    3 cltbld   cltbld        102 Mar 12 19:06 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin
1809784007    0 drwxr-xr-x    4 cltbld   cltbld        136 Mar 18 09:32 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents
1809784008    0 drwxr-xr-x    2 cltbld   cltbld         68 Mar 18 09:32 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents/MacOS
1809784009    0 drwxr-xr-x    3 cltbld   cltbld        102 Mar 12 19:06 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents/Resources
1809784010    0 drwxr-xr-x    3 cltbld   cltbld        102 Mar 18 09:32 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents/Resources/English.lproj
1809784011    0 drwxr-xr-x    2 cltbld   cltbld         68 Mar 18 09:32 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents/Resources/English.lproj/PrintPDE.nib

Some empty directories (which the updater can't remove) and the harmless comment change.

Windows

 Only in source/bin: .autoreg
 diff -r source/bin/defaults/pref/channel-prefs.js target/bin/defaults/pref/channel-prefs.js
 1c1
 < //@line 2 "e:\fx19rel\WINNT_5.2_Depend\mozilla\browser\app\profile\channel-prefs.js"
 ---
 > //@line 2 "e:\builds\moz2_slave\win32_build\build\browser\app\profile\channel-prefs.js"
 Binary files source/bin/freebl3.chk and target/bin/freebl3.chk differ
 Binary files source/bin/nssdbm3.chk and target/bin/nssdbm3.chk differ
 Only in source/bin: removed-files

Binary files source/bin/softokn3.chk and target/bin/softokn3.chk differ 3x chk files are expected (until bug 489961 is resolved), .autoreg and removed-files are OK, and harmless comment change in channel-prefs.js

Push updates to beta channel

# cltbld@aus2-staging.m.o
# make sure scripts are up to date
cd bin
cvs update
~/bin/backupsnip 20100315-Firefox-3.0.19-beta
~/bin/pushsnip 20100315-Firefox-3.0.19-beta

Sign Installers

  • Done manually on the day before release using these installer-signing-instructions here

On Stage complete stage-merged:

# cltbld@stage
cd /data/cltbld/firefox-3.0.19/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/

Create MD5 and SHA1 checksum files

# on stage
cd /data/cltbld/firefox-3.0.19/stage-merged/
~/bin/checksum-files .

Fix permissions & ownership (on the two SUM files, and the detached sigs)

chown -R cltbld:firefox .
chmod 644 *SUMS

Update Bouncer

Added with bouncer buddy, including solaris entries for tarballs and complete updates.

Push to Mirrors

  • push the stage-merged directory to the releases area:
# ffxbld@stage
rsync -av /data/cltbld/firefox-3.0.19/stage-merged/ /home/ftp/pub/firefox/releases/3.0.19/
  • edit the exclude file (as cltbld) /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to add the new release (3.0.19) and remove the previous release (3.0.18).


Final Verification

  • on fx-linux-1.9-slave2.build
hg clone http://hg.mozilla.org/build/tools
cd tools/release/updates
cat moz19-firefox-{win32,mac,linux}.cfg | grep -v major | sed 's/betatest/releasetest/' > update.cfg
./verify.sh -t update.cfg 2>&1 | tee quickVerify.log

Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):

grep HTTP quickVerify.log | grep -v 200 | grep -v 302

Repeat for major update files

cat moz19-firefox-{win32,mac,linux}-major.cfg | grep -v major | sed 's/betatest/releasetest/' > update-mu.cfg
./verify.sh -t update-mu.cfg 2>&1 | tee quickVerify-MU.log

Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):

grep HTTP quickVerify-MU.log | grep -v 200 | grep -v 302

Before pushing final updates, verify that "release" and "releasetest" channel match:

# on aus2-staging
cd /opt/aus2/snippets/staging/20100315-Firefox-3.0.19
find -type d -iregex '.*release.*' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20100315-Firefox-3.0.19-test/$a");'

Publish Updates to Release Channel

Since bug 508406 we should run backupsnip every time we push for a given code branch, and not rely on it being run elsewhere. This will take about 10 minutes to run and can be done ahead of time (in a screen session):

~/bin/backupsnip 20100315-Firefox-3.0.19


After go email from driver:

~/bin/pushsnip   20100315-Firefox-3.0.19

Provided all the billboards are done we can enable the major update

~/bin/backupsnip 20100317-Firefox-3.0.19-3.6.2-MU-beta
~/bin/pushsnip   20100317-Firefox-3.0.19-3.6.2-MU
~/bin/pushsnip   20100317-Firefox-3.0.19-3.6.2-MU-beta

Release

# ffxbld@stage
cd /home/ftp/pub/firefox/releases
rm latest-3.0 && ln -s 3.0.19 latest-3.0

Post-Release Issues

Because I missed a step in the signing of installers, namely SIGNING the win32 INSTALLERS, bug 556222 was filed and I signed the windows installers according to the steps in the signing doc, then synched those newly signed win32 installers to the ftp release directory.

The builds were pushed back to stage wrong, and ended up in /data/cltbld/firefox-3.0.19/stage-merged/win32/win32. They were then pushed to /releases/ wrong by doing:

rsync -av /data/cltbld/firefox-3.0.19/stage-merged/ /home/ftp/pub/firefox/releases/3.0.19/

...which created a win32/win32 directory there, too.

Fixed all of this by doing:

# cltbld @ stage
rsync -av /data/cltbld/firefox-3.0.19/stage-merged/win32/win32/ /home/ftp/pub/firefox/releases/3.0.19/win32/
rm -rf /data/cltbldl/firefox-3.0.19/stage-merged/win32/win32
rm -rf /home/ftp/pub/firefox/releases/3.0.19/win32/win32
rsync -av /data/cltbld/firefox-3.0.19/stage-merged/win32/ /home/ftp/pub/firefox/releases/3.0.19/win32/

...and then we realized that we can't just override files on stage, because we can't guarantee that all of the mirrors will pick them up. The sections below track fixing that.

Push to mirrors (again)

The win32 builds were pushed out first to get the mirrors syncing them first.

# ffxbld@stage
mkdir -p /home/ftp/pub/firefox/releases/3.0.19-real/win32
rsync -av /data/cltbld/firefox-3.0.19/stage-merged/win32/ /home/ftp/pub/firefox/releases/3.0.19-real/win32/
rsync -av /data/cltbld/firefox-3.0.19/stage-merged/ /home/ftp/pub/firefox/releases/3.0.19-real/
rsync -av /home/ftp/pub/firefox/releases/3.0.19/contrib* /home/ftp/pub/firefox/releases/3.0.19-real/

After both finished, updated /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to replace '3.0.19' with '3.0.19-real'

Added a temporary bouncer entry 'bhearsum-test' that points to /firefox/releases/3.0.19-real/update/win32/en-US/firefox-3.0.19.complete.mar (because that was the last file pushed).

Update Bouncer Entries

After the temporary bouncer entry is showing as much uptake as the real 3.0.19 entries, they can be flipped over to 3.0.19-real. We can't do this beforehand because it takes Bouncer/Sentry some time to update its valid mirror list.
This was done around 16:30 PDT on March 31, 2010.

Delete original 3.0.19

The following day (April 1st) the 3.0.19 directory was deleted in order to avoid confusion:

# ffxbld@stage
cd /home/ftp/pub/firefox/releases
rm latest-3.0 && ln -s 3.0.19-real latest-3.0
rm -rf 3.0.19

3.0.19 -> 3.6.3 Major Update Refresh

Snippet Generation

on fx-linux-1.9-slave2:

mkdir -p /builds/3.0.19-3.6.3-major-update/snippets
cd /builds/3.0.19-3.6.3-major-update/snippets

cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.6.3 3.0.19-3.6.3
cd ../..

# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
Quick Verify

Check that releasetest = beta = release

cd temp/firefox/3.0.19-3.6.3

# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

# beta == release
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/aus2\.beta/aus2/; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/aus2/aus2.beta/; system("diff -r -u $_ $a");'
Push to AUS
cd /builds/3.0.19-3.6.3-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.3
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100402-Firefox-3.0.19-3.6.3-MU-test
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100402-Firefox-3.0.19-3.6.3-MU-beta
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/      cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100402-Firefox-3.0.19-3.6.3-MU
Enable test snippets
ssh -i ~/.ssh/aus2 cltbld@aus2-staging.mozilla.org

~/bin/backupsnip 20100402-Firefox-3.0.19-3.6.3-MU-test
~/bin/pushsnip   20100402-Firefox-3.0.19-3.6.3-MU-test
Update Verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

 mkdir -p /builds/verify/firefox-3019-363-major/
 cd /builds/verify/firefox-3019-363-major/
 hg clone http://hg.mozilla.org/build/tools
 
 cd tools/release/common
 cd ../updates
 platform=linux   # or mac or win32
 ./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log
Results

Linux

diff -r source/firefox/defaults/pref/channel-prefs.js target/firefox/defaults/pref/channel-prefs.js

Harmless conmment change

Locale-specific: leftover searchplugins

  • fr
Only in source/firefox/searchplugins: MediaDICO-fr.xml
  • hu
Only in source/firefox/searchplugins: amazondotcom.xml
Only in source/firefox/searchplugins: answers.xml
Only in source/firefox/searchplugins: eBay.xml
Only in source/firefox/searchplugins: yahoo.xml
  • it
Only in source/firefox/searchplugins: demauro.xml
  • sl
Only in source/firefox/searchplugins: iskalnik-gov-si.xml
Only in source/firefox/searchplugins: najdi-si.xml
Only in source/firefox/searchplugins: yahoo.xml
  • uk
Only in source/firefox/searchplugins: price-ua.xml

Locale-specific: no locale for 3.6.2

  • mn
FAIL: no partial update found for https://aus2.mozilla.org/update/1/Firefox/3.0.19/2010031218/Linux_x86-gcc3/mn/betatest/update.xml?force=1
FAIL: download_mars returned non-zero exit code: 1

Mac

diff -r source/Firefox.app/Contents/MacOS/defaults/pref/channel-prefs.js target/Firefox.app/Contents/MacOS/defaults/pref/channel-prefs.js
1c1
< //@line 2 "/builds/tinderbox/Fx-Mozilla1.9-Release/Darwin_8.8.4_Depend/mozilla/browser/app/profile/channel-prefs.js"
---
> //@line 2 "/builds/slave/macosx_build/build/browser/app/profile/channel-prefs.js"
Only in source/Firefox.app/Contents/MacOS/plugins: Default Plugin.plugin
Only in source/Firefox.app/Contents/MacOS/res/cursors: CVS
Only in source/Firefox.app/Contents: Plug-Ins
Contents of source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin dir only in source or target
1823761972    0 drwxr-xr-x    4 cltbld   cltbld        136 Apr  2 15:09 source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin/Contents
1823761973    0 drwxr-xr-x    2 cltbld   cltbld         68 Apr  2 15:09 source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin/Contents/MacOS
1823761974    0 drwxr-xr-x    3 cltbld   cltbld        102 Apr  2 15:09 source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin/Contents/Resources
1823761975    0 drwxr-xr-x    2 cltbld   cltbld         68 Apr  2 15:09 source/Firefox.app/Contents/MacOS/plugins/Default Plugin.plugin/Contents/Resources/English.lproj
Contents of source/Firefox.app/Contents/MacOS/res/cursors/CVS dir only in source or target
Contents of source/Firefox.app/Contents/Plug-Ins dir only in source or target
1823762005    0 drwxr-xr-x    3 cltbld   cltbld        102 Mar 12 19:06 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin
1823762006    0 drwxr-xr-x    4 cltbld   cltbld        136 Apr  2 15:09 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents
1823762007    0 drwxr-xr-x    2 cltbld   cltbld         68 Apr  2 15:09 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents/MacOS
1823762008    0 drwxr-xr-x    3 cltbld   cltbld        102 Mar 12 19:06 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents/Resources
1823762009    0 drwxr-xr-x    3 cltbld   cltbld        102 Apr  2 15:09 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents/Resources/English.lproj
1823762010    0 drwxr-xr-x    2 cltbld   cltbld         68 Apr  2 15:09 source/Firefox.app/Contents/Plug-Ins/PrintPDE.plugin/Contents/Resources/English.lproj/PrintPDE.nib


Some empty directories (which the updater can't remove) and the harmless comment change.

Windows

 Only in source/bin: .autoreg
 diff -r source/bin/defaults/pref/channel-prefs.js target/bin/defaults/pref/channel-prefs.js
 1c1
 < //@line 2 "e:\fx19rel\WINNT_5.2_Depend\mozilla\browser\app\profile\channel-prefs.js"
 ---
 > //@line 2 "e:\builds\moz2_slave\win32_build\build\browser\app\profile\channel-prefs.js"
 Binary files source/bin/freebl3.chk and target/bin/freebl3.chk differ
 Binary files source/bin/nssdbm3.chk and target/bin/nssdbm3.chk differ
 Only in source/bin: removed-files
 Binary files source/bin/softokn3.chk and target/bin/softokn3.chk differ

3x chk files are expected (until bug 489961 is resolved), .autoreg and removed-files are OK, and harmless comment change in channel-prefs.js

Publish updates to release channel

~/bin/backupsnip 20100402-Firefox-3.0.19-3.6.3-MU-beta
~/bin/pushsnip   20100402-Firefox-3.0.19-3.6.3-MU
~/bin/pushsnip   20100402-Firefox-3.0.19-3.6.3-MU-beta

One more time with signing

April 12, 2010 bug 557463 pointed out that the signatures were older than the signed installers. I re-generated the pgp signatures, SUMS, and synched to 3.0.19-real

Push to mirrors (again)

The win32 builds were pushed out first to get the mirrors syncing them first.

# ffxbld@stage
mkdir -p /home/ftp/pub/firefox/releases/3.0.19-real-real/win32
rsync -av /data/cltbld/firefox-3.0.19/stage-merged/win32/ /home/ftp/pub/firefox/releases/3.0.19-real-real/win32/
rsync -av /data/cltbld/firefox-3.0.19/stage-merged/ /home/ftp/pub/firefox/releases/3.0.19-real-real/
rsync -av /home/ftp/pub/firefox/releases/3.0.19-real/contrib* /home/ftp/pub/firefox/releases/3.0.19-real-real/

After both finished, updated /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to replace '3.0.19-real' with '3.0.19-real-real'

Added a temporary bouncer entry 'lsblakk-test' that points to /firefox/releases/3.0.19-real-real/update/win32/en-US/firefox-3.0.19.complete.mar (because that was the last file pushed).

Update Bouncer Entries

After the temporary bouncer entry is showing as much uptake as the 3.0.19-real entries, they can be flipped over to 3.0.19-real-real. We can't do this beforehand because it takes Bouncer/Sentry some time to update its valid mirror list.
This was done around 11:55 PDT on April 13, 2010.

Delete 3.0.19-real

The following day (April 14th) the 3.0.19-real directory was deleted in order to avoid confusion:

# ffxbld@stage
cd /home/ftp/pub/firefox/releases
rm latest-3.0 && ln -s 3.0.19-real-real latest-3.0
rm -rf 3.0.19-real