Checklist

Fennec

~~(RelMan) Ship the l10n milestone~~

(RelMan) Submit to Ship It

Push to mirrors

Post-release tasks

~~Mark as shipped~~

Firefox

~~(RelMan) Ship the l10n milestone~~

(RelMan) Submit to Ship It

Publish RC to Beta channel

Push to mirrors

Apply Throttling

Verify throttling

Run pushsnip

Post-release tasks

~~Mark as shipped~~

Build 1

RC

Updates

on b-linux64-hp-0020

mkdir bug1118476
cd bug1118476

hg clone http://hg.mozilla.org/build/tools
wget -O repack.py https://bugzilla.mozilla.org/attachment.cgi?id=8433787
export PYTHONPATH=tools/lib/python; export PATH=$PATH:`pwd`
# make sure enough disk space is free
python ./tools/buildfarm/maintenance/purge_builds.py -s 55 -n 'rel-*:45d' -n 'tb-rel-*:45d' /builds/slave

# get the token to ./token

# in 4 different screen windows run
cd ~/bug1118476
export PYTHONPATH=tools/lib/python; export PATH=$PATH:`pwd`
time python -u repack.py -f 35.0b8-build1 -t 35.0-build1 -T 35.0b8-build1 -P linux-x86_64 2>&1 | tee linux64.log
time python -u repack.py -f 35.0b8-build1 -t 35.0-build1 -T 35.0b8-build1 -P linux-i686 --no-env 2>&1 | tee linux.log
time python -u repack.py -f 35.0b8-build1 -t 35.0-build1 -T 35.0b8-build1 -P mac --no-env 2>&1 | tee mac.log
time python -u repack.py -f 35.0b8-build1 -t 35.0-build1 -T 35.0b8-build1 -P win32 --no-env 2>&1 | tee win32.log

# fix permissions and upload to stage
find repacked/ -type f -exec chmod 644 {} \;
find repacked/ -type d -exec chmod 755 {} \;
# rsync -n is your friend
rsync -av -e 'ssh -i /home/cltbld/.ssh/ffxbld_rsa' \
  repacked/35.0-candidates/build1/ \
  ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/candidates/35.0-candidates/build1/
# reran check_permissions builder to make sure all is correct.
# forced checksum generation. failed as expected. the builder updated the SHA512SUM file only.
# in case we want to use checksum to submit data to balrog

# run av scan on partials, anyone on stage
cd /pub/mozilla.org/firefox/candidates/35.0-candidates/build1/update
nice ionice -c2 -n7 extract_and_run_command.py -j2 clamdscan -m --no-summary -- \
  `find . -name firefox-35.0b8-35.0.partial.mar` 2>&1 | tee /tmp/av.log

Fennec postrelease

Push to mirrors was missing from the Fennec checklist (copied beta one by mistake??), so post_release failed with

ERROR: push to mirrors directory, /pub/mozilla.org/mobile/releases/35.0/, does not exist on stage.mozilla.org
ERROR: Did you push to mirrors before running post release?

Did mirror push, then post release.

Build 2

~~(RelMan) Ship the l10n milestone~~

~~(RelMan) Submit to Ship It~~

~~At the same time, run pushsnip and publish in Balrog~~
- see below for issue caused by pushing wrong snippet set
Post-release tasks
Mark as shipped

Updates for beta/beta-cdntest

Created firefox-35.0build2-complete and firefox-35.0build2-partial-35.0b8 bouncer entries
- Locations for linux, linux64, win32, and osx, pointing at the candidates directory.
Partial MARs were generated at build time, no need to do anything there.
Added the following block to the fileUrls section of the Firefox-35.0-build2 release in Balrog:

+    },
+    "beta": {
+      "partials": {
+        "Firefox-35.0b8-build1": "http://download.mozilla.org/?product=firefox-35.0build2-partial-35.0b8&os=%OS_BOUNCER%&lang=%LOCALE%"
+      },
+      "completes": {
+        "*": "http://download.mozilla.org/?product=firefox-35.0build2-complete&os=%OS_BOUNCER%&lang=%LOCALE%"
+      }
+    },
+    "beta-cdntest": {
+      "partials": {
+        "Firefox-35.0b8-build1": "http://download.mozilla.org/?product=firefox-35.0build2-partial-35.0b8&os=%OS_BOUNCER%&lang=%LOCALE%"
+      },
+      "completes": {
+        "*": "http://download.mozilla.org/?product=firefox-35.0build2-complete&os=%OS_BOUNCER%&lang=%LOCALE%"
+      }

Updated it via the new admin ui on aus4-admin.mozilla.org.
Updated the Firefox beta-cdntest channel rule to point at Firefox-35.0-build2.

Update Verify

On bhearsum's laptop, create configs:

cd <parent_of_tools_checkout, updated to tip>
export PERL5LIB="tools/lib/perl:$HOME/perl5/lib/perl5"  # need Config::General v2.44
wget http://hg.mozilla.org/releases/mozilla-release/raw-file/FIREFOX_35_0_RELEASE/browser/locales/shipped-locales
perl tools/release/patcher-config-bump.pl -p firefox -r Firefox -v 35.0 -a 35.0 -o 35.0b8 -b 2 \
  -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg -t stage.mozilla.org -f ftp.mozilla.org \
  -d download.mozilla.org -l shipped-locales --partial-version 35.0b8 --platform linux --platform linux64 \
  --platform macosx64 --platform win32

# generate update verify configs
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux64 --output tools/release/updates/mozBeta-firefox-linux64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_35_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux --output tools/release/updates/mozBeta-firefox-linux.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_35_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform macosx64 --output tools/release/updates/mozBeta-firefox-mac64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_35_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform win32 --output tools/release/updates/mozBeta-firefox-win32.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_35_0_RELEASE

hg -R tools/ diff tools/release/updates > patch

Attached to bug 1118476.

To run the tests:

# linux32 setup; others don't need anything
sh -c 'rm -f /builds/mock_mozilla/mozilla-centos6-x86_64/buildroot.lock; mock_mozilla -r mozilla-centos6-x86_64 --orphanskill'
mock_mozilla -r mozilla-centos6-x86_64 --init
mock_mozilla -r mozilla-centos6-x86_64 --install autoconf213 python zip mozilla-python27-mercurial git ccache glibc-static.i686 libstdc++-static.i686 perl-Test-Simple perl-Config-General gtk2-devel.i686 libnotify-devel.i686 yasm alsa-lib-devel.i686 libcurl-devel.i686 wireless-tools-devel.i686 libX11-devel.i686 libXt-devel.i686 mesa-libGL-devel.i686 gnome-vfs2-devel.i686 GConf2-devel.i686 wget mpfr xorg-x11-font* imake gcc45_0moz3 gcc454_0moz1 gcc472_0moz1 gcc473_0moz1 yasm ccache valgrind pulseaudio-libs-devel.i686 gstreamer-devel.i686 gstreamer-plugins-base-devel.i686 glibc-devel.i686 libgcc.i686 libstdc++-devel.i686 ORBit2-devel.i686 atk-devel.i686 cairo-devel.i686 check-devel.i686 dbus-devel.i686 dbus-glib-devel.i686 fontconfig-devel.i686 glib2-devel.i686 hal-devel.i686 libICE-devel.i686 libIDL-devel.i686 libSM-devel.i686 libXau-devel.i686 libXcomposite-devel.i686 libXcursor-devel.i686 libXdamage-devel.i686 libXdmcp-devel.i686 libXext-devel.i686 libXfixes-devel.i686 libXft-devel.i686 libXi-devel.i686 libXinerama-devel.i686 libXrandr-devel.i686 libXrender-devel.i686 libXxf86vm-devel.i686 libdrm-devel.i686 libidn-devel.i686 libpng-devel.i686 libxcb-devel.i686 libxml2-devel.i686 pango-devel.i686 perl-devel.i686 pixman-devel.i686 zlib-devel.i686 freetype-2.3.11-6.el6_1.8.i686 freetype-devel-2.3.11-6.el6_1.8.i686 freetype-2.3.11-6.el6_1.8.x86_64
mock_mozilla -r mozilla-centos6-x86_64 --cwd /builds/slave --unpriv --shell 'PATH="/tools/buildbot/bin:/usr/local/bin:/usr/lib/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin" bash'
# end linux32 setup

mkdir bug1118476 ; cd $_
hg clone http://hg.mozilla.org/build/tools
cd tools
wget -O- https://bug1118476.bugzilla.mozilla.org/attachment.cgi?id=8545361 | patch -p1 
cd release/updates

On b-linux64-hp-0020 see screen 0 for linux32 & screen 1 for linux64; screen on bld-lion-r5-017; vnc on b-2008-ix-0018 Adjust to right config for each platform (linux64 vs linux vs mac64 vs win32)

./verify.sh -c mozBeta-firefox-linux64.cfg 2>&1 | tee log

Results - expected difference in channel and ACCEPTED_MAR_CHANNEL_IDS

non-fatal error removing directory: components/, rv: 0, err: 39
succeeded
calling QuitProgressUI
diff -r source/firefox/defaults/pref/channel-prefs.js target/firefox/defaults/pref/channel-prefs.js
5c5
< pref("app.update.channel", "beta");
---
> pref("app.update.channel", "release");
diff -r source/firefox/update-settings.ini target/firefox/update-settings.ini
5c5
< ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-beta,firefox-mozilla-release
---
> ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-release
WARN: non-binary files found in diff
WARN: check_updates returned warning for Linux_x86_64-gcc3 downloads/firefox-35.0b8.tar.bz2 vs. downloads/firefox-35.0.tar.bz2: 2

Note: we shipped prior to anyone verifying the update verify logs

issues

repack 8/10 linux64:

command timed out: 2400 seconds without output running [...]

during the upload of pl binaries, full log here. Job re-triggered

Updates

The release snippets were pushed too early instead of pushing the release builds to the beta users via balrog.
Stop the end user impact:
- chmod 700 the Firefox dir on aus3-staging (disables all updates)
- Updated the rules in balrog to serve 35.0-build2 to beta users
Correct things:
- untar correct updates from backup 20150107-nightly-1.tar.bz2 into /opt/aus2/snippets/staging/Firefox-35.0-recovery
- move asside all releases before 10.0 as they weren't impacted (into the old directory
- restore with rsync the correct data to production using ./pushit.sh
- restored the permissions on the Firefox dir
- tested a few updates from 10.0 and 33 - got correct results of update to 34.0.5
- repushed the Firefox-35.0-build2-test snippets (not in above backup)

Build 3

release-runner failed because ship-it comments contained unicode symbols. A temporary fix (applied locally) in bug 1119638

Updates for beta/beta-cdntest

Created firefox-35.0build3-complete and firefox-35.0build3-partial-35.0b8 bouncer entries
- Locations for linux, linux64, win32, and osx, pointing at the candidates directory.
Partial MARs were generated at build time, no need to do anything there.
Added the following block to the fileUrls section of the Firefox-35.0-build3 release in Balrog:

+    },
+    "beta": {
+      "partials": {
+        "Firefox-35.0b8-build1": "http://download.mozilla.org/?product=firefox-35.0build3-partial-35.0b8&os=%OS_BOUNCER%&lang=%LOCALE%"
+      },
+      "completes": {
+        "*": "http://download.mozilla.org/?product=firefox-35.0build3-complete&os=%OS_BOUNCER%&lang=%LOCALE%"
+      }
+    },
+    "beta-cdntest": {
+      "partials": {
+        "Firefox-35.0b8-build1": "http://download.mozilla.org/?product=firefox-35.0build3-partial-35.0b8&os=%OS_BOUNCER%&lang=%LOCALE%"
+      },
+      "completes": {
+        "*": "http://download.mozilla.org/?product=firefox-35.0build3-complete&os=%OS_BOUNCER%&lang=%LOCALE%"
+      }

Updated it via the new admin ui on aus4-admin.mozilla.org.
Updated the Firefox beta-cdntest channel rule to point at Firefox-35.0-build3.

Update Verify

Note: These will probably not run to completion before we ship.

On bhearsum's laptop, create configs:

cd <parent_of_tools_checkout, updated to tip>
export PERL5LIB="tools/lib/perl:$HOME/lib/perl"  # need Config::General v2.44
wget http://hg.mozilla.org/releases/mozilla-release/raw-file/FIREFOX_35_0_RELEASE/browser/locales/shipped-locales
perl tools/release/patcher-config-bump.pl -p firefox -r Firefox -v 35.0 -a 35.0 -o 35.0b8 -b 3 \
  -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg -t stage.mozilla.org -f ftp.mozilla.org \
  -d download.mozilla.org -l shipped-locales --partial-version 35.0b8 --platform linux --platform linux64 \
  --platform macosx64 --platform win32

# generate update verify configs
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux64 --output tools/release/updates/mozBeta-firefox-linux64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_35_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux --output tools/release/updates/mozBeta-firefox-linux.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_35_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform macosx64 --output tools/release/updates/mozBeta-firefox-mac64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_35_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform win32 --output tools/release/updates/mozBeta-firefox-win32.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_35_0_RELEASE

hg -R tools/ diff tools/release/updates > patch

Attached to bug 1118476.

To run the tests:

# linux32 setup; others don't need anything
sh -c 'rm -f /builds/mock_mozilla/mozilla-centos6-x86_64/buildroot.lock; mock_mozilla -r mozilla-centos6-x86_64 --orphanskill'
mock_mozilla -r mozilla-centos6-x86_64 --init
mock_mozilla -r mozilla-centos6-x86_64 --install autoconf213 python zip mozilla-python27-mercurial git ccache glibc-static.i686 libstdc++-static.i686 perl-Test-Simple perl-Config-General gtk2-devel.i686 libnotify-devel.i686 yasm alsa-lib-devel.i686 libcurl-devel.i686 wireless-tools-devel.i686 libX11-devel.i686 libXt-devel.i686 mesa-libGL-devel.i686 gnome-vfs2-devel.i686 GConf2-devel.i686 wget mpfr xorg-x11-font* imake gcc45_0moz3 gcc454_0moz1 gcc472_0moz1 gcc473_0moz1 yasm ccache valgrind pulseaudio-libs-devel.i686 gstreamer-devel.i686 gstreamer-plugins-base-devel.i686 glibc-devel.i686 libgcc.i686 libstdc++-devel.i686 ORBit2-devel.i686 atk-devel.i686 cairo-devel.i686 check-devel.i686 dbus-devel.i686 dbus-glib-devel.i686 fontconfig-devel.i686 glib2-devel.i686 hal-devel.i686 libICE-devel.i686 libIDL-devel.i686 libSM-devel.i686 libXau-devel.i686 libXcomposite-devel.i686 libXcursor-devel.i686 libXdamage-devel.i686 libXdmcp-devel.i686 libXext-devel.i686 libXfixes-devel.i686 libXft-devel.i686 libXi-devel.i686 libXinerama-devel.i686 libXrandr-devel.i686 libXrender-devel.i686 libXxf86vm-devel.i686 libdrm-devel.i686 libidn-devel.i686 libpng-devel.i686 libxcb-devel.i686 libxml2-devel.i686 pango-devel.i686 perl-devel.i686 pixman-devel.i686 zlib-devel.i686 freetype-2.3.11-6.el6_1.8.i686 freetype-devel-2.3.11-6.el6_1.8.i686 freetype-2.3.11-6.el6_1.8.x86_64
mock_mozilla -r mozilla-centos6-x86_64 --cwd /builds/slave --unpriv --shell 'PATH="/tools/buildbot/bin:/usr/local/bin:/usr/lib/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin" bash'
# end linux32 setup

mkdir bug1118476 ; cd $_
hg clone http://hg.mozilla.org/build/tools
cd tools
wget -O- https://bug1118476.bugzilla.mozilla.org/attachment.cgi?id=8546626 | patch -p1 
cd release/updates

On b-linux64-hp-0023 see screen 0 for linux32 & screen 1 for linux64; screen on bld-lion-r5-003; vnc on b-2008-ix-0079 Adjust to right config for each platform (linux64 vs linux vs mac64 vs win32)

./verify.sh -c mozBeta-firefox-linux64.cfg 2>&1 | tee log

Results - expected difference in channel and ACCEPTED_MAR_CHANNEL_IDS

non-fatal error removing directory: components/, rv: 0, err: 39
succeeded
calling QuitProgressUI
diff -r source/firefox/defaults/pref/channel-prefs.js target/firefox/defaults/pref/channel-prefs.js
5c5
< pref("app.update.channel", "beta");
---
> pref("app.update.channel", "release");
diff -r source/firefox/update-settings.ini target/firefox/update-settings.ini
5c5
< ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-beta,firefox-mozilla-release
---
> ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-release
WARN: non-binary files found in diff
WARN: check_updates returned warning for Linux_x86_64-gcc3 downloads/firefox-35.0b8.tar.bz2 vs. downloads/firefox-35.0.tar.bz2: 2

Adjust whatsnew page

Per bug 1120498, we want a whatsnew page for 33.0.2 and earlier, no whatsnew for 34.0 and 34.0.5. No snippets have whatsnew in them yet, so we need to add it for all of the releases that need it (and do nothing for 34.0 and 34.0.5):

# ffxbld@aus3-staging
# test snippets
cd /opt/aus2/snippets/staging
rsync -a --exclude=betatest --exclude=35.0 --exclude=34.0 --exclude=34.0.5 /opt/aus2/snippets/pushed/Firefox-35.0-build3-test/ Firefox-35.0-build3-test-whatsnew/
for i in `find Firefox-35.0-build3-test-whatsnew/ -type f`; do locale=`echo $i | cut -d/ -f6` && echo "actions=showURL" >> $i && echo "openURL=https://www.mozilla.org/$locale/firefox/35.0/whatsnew/?oldversion=%OLD_VERSION%" >> $i; done
# live snippets
# back them up first
rsync -a Firefox-35.0-build3/ /opt/aus2/snippets/backup/Firefox-35.0-build3.bak/
for i in `find Firefox-35.0-build3/ -type f -not -wholename "*/35.0/*" -not -wholename "*/34.0/*" -not -wholename "*/34.0.5/*"`; do locale=`echo $i | cut -d/ -f6` && echo "actions=showURL" >> $i && echo "openURL=https://www.mozilla.org/$locale/firefox/35.0/whatsnew/?oldversion=%OLD_VERSION%" >> $i; done

# Verify that releasetest = release
# Fails about missing 34.0, 34.0.5, and 35.0 snippets are OK because we didn't regenerate those in the test snippets side.
hg clone https://hg.mozilla.org/build/tools ~/
bash python ~/tools/release/compare-channel-snippets.sh Firefox-35.0-build3 release /opt/aus2/snippets/pushed/Firefox-35.0-build3-test-whatsnew releasetest | tee ~/verify.log

Revert bug 1110301 partner update blocks

After confirming with mconnor that we should give partners updates again, did the following to unblock them:

# ffxbld@aus3-staging
cd /opt/aus2/incoming/3/Firefox
# did some trial runs without -exec to make sure only the right dirs would be found
find . -mindepth 5 -maxdepth 5 -not -wholename "*1.5*" -not -wholename "*2.0.0*" -not -wholename "*3.6*" -name "*cck*" -exec rm -rf {} \;

Releases/Firefox 35.0/BuildNotes

Contents

Checklist

Fennec

Firefox

Build 1

RC

Updates

Fennec postrelease

Build 2

Updates for beta/beta-cdntest

Update Verify

issues

Updates

Build 3

Updates for beta/beta-cdntest

Update Verify

Adjust whatsnew page

Revert bug 1110301 partner update blocks

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

How to Contribute

MozillaWiki

Around Mozilla

Tools