Releases/Fennec 1.1/BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineers

aki - bug 566428

Signed-off Revision(s)

Build # mozilla-1.9.2 mobile-1.1
rc1 build 1 2cb83e106cf7 4a51427bed7c
rc1 build 2 91e79f7e8b96 4a51427bed7c
rc1 build 3 91e79f7e8b96 5c06d8f81a1a

Tags

On releases/mozilla-1.9.2:

Build # Tag Moz Changeset Mobile Changeset
rc1 build 1 GECKO1925_20100602_RELBRANCH 2cb83e106cf7 4a51427bed7c
FENNEC_1_1rc1_BUILD1 7275c58eba49 c6ad008b32ad
rc1 build 2 FENNEC_1_1rc1_BUILD2 91e79f7e8b96 c6ad008b32ad
rc1 build 3 FENNEC_1_1rc1_BUILD3 91e79f7e8b96 5c06d8f81a1a

Build data

Build # Type Build ID Build machine
rc1 build 1 Maemo4
Linux 20100602181702 moz2-linux-slave35
Mac 20100602184129 moz2-darwin9-slave49
Windows 20100602185328 win32-slave59
rc1 build 2 Maemo4 20100603130022 moz2-linux-slave40
Linux 20100603125742 moz2-linux-slave43
Mac 20100603131037 moz2-darwin9-slave30
Windows 20100603123420 mw32-ix-slave22
rc1 build 3 Maemo4 20100614152856 moz2-linux-slave37
Linux 20100614142409 mv-moz2-linux-ix-slave02
Mac 20100614152130 moz2-darwin9-slave30
Windows 20100615115055 mw32-ix-slave23

Notes

Build 1

  • Wished we had a clobberer for mobile releases bug 537426
  • Axel landed the l10n json update
  • Landed a release config update which contains changesets and turns mergeLocales to False (no English strings in l10n repacks)
  • Reconfiged pm02
  • Started the automation with:
buildbot sendchange --username=aki --master=localhost:9010 --branch=releases/mobile-1.1 "Fennec 1.1rc1 build1" doit

Tag

Timed out after 1800 seconds during nl clone. Following the docs, but editing l10n-changesets_mobile-1.1.json .

  • Copied l10n-changesets_mobile-1.1.json to l10n-changesets_mobile-1.1.json.orig
  • Edited a copy of l10n-changesets_mobile-1.1.json (aki.json, softlinked to l10n-changesets_mobile-1.1.json) to remove the already tagged locales (ar through nb-NO)
  • Tried a "force build" of the mobile_tag builder, which failed on ar
  • Reconfiged pm02
  • Tried another "force build" of the mobile_tag builder
  • That worked; reverted l10n-changesets_mobile-1.1.json changes
  • Followed the docs on creating a dummy tag_factory for tagging in release_mobile_master.py
  • Reconfiged pm02
  • Restarted automation with
buildbot sendchange --username=aki --master=localhost:9010 --branch=releases/mobile-1.1 "Fennec 1.1rc1 build1" doit
  • Started scheming on how to make this better
  • While filling out the tags, noticed that bug 542413 bit me (further maemo release bumpfiles support) -- no milestone update
  • Killed builds
  • Manually updated the milestone in bug 566428 and tagged via
hg tag -f -r 7275c58eba49 FENNEC_1_1rc1_BUILD1
hg tag -f -r 7275c58eba49 FENNEC_1_1rc1_RELEASE
hg push
  • Verified dummy factory still there (wasn't, due to hg pull -u for bug 542413; replaced)
  • Manually clobbered:
    • moz2-linux-slave39:/builds/slave/mobile_source
    • mv-moz2-linux-ix-slave14:/scratchbox/users/cltbld/home/cltbld/build/maemo-1.9.2-release
    • mv-moz2-linux-ix-slave24:/builds/slave/mobile_linux-i686_desktop_build
    • moz2-darwin9-slave57:/builds/slave/mobile_macosx-i686_desktop_build
    • win32-slave46:/e/builds/moz2_slave/mobile_win32-i686_desktop_build
  • Restarted automation via sendchange
  • Gah, dummy step didn't take. Double checked again, reconfiged again after |rm *.pyc|
  • sendchange #4
  • fixed typo :( ... reconfig, sendchange #5

Source

No problems.

(Source built 2x due to restart; was fine both times. I nuked stage's build1 candidates dir in between to be sure.)

(Killed mobile_source build 20 when it was triggered from a sendchange in the build/repack steps)

Build/Repack

  • Maemo4: ran into bug 567827 which killed the build before the multilocale repack portion.
  • Linux: no problems
  • Mac: no problems
  • Win32: no problems

Maemo4 needs to succeed during a sendchange since repacks are triggered from it. The desktop builds have no repacks to worry about.

So, to fix maemo4, I:

  • edited mobile_config.py MOBILE_SLAVES to point maemo4 to config.LINUX_VMS (bug 567827 kills make buildsymbols on ix boxes only).
  • linux+mac were done and win32 putting away happily, so I added a dummy build_factory to the |for platform in enUSDesktopPlatforms:| loop in release_mobile_master.py
  • reconfiged
  • sendchange #6?
  • removed maemo4 from candidates/1.1rc1-candidates/build1/ just to be sure
  • Gah, noticed that bug 566311 isn't fixed on m-1.9.2; flagged a1.9.2.5?

Partner Builds

  • Didn't trigger for some reason :P
    • Force built, worked.

Deb Repo Creation

Push Live

Copy bits to releases/
Create new .install files
Create bouncer entries

Build 2

  • Wished we had a clobberer for mobile releases bug 537426
    • clobbered mv-moz2-linux-ix-slave22, moz2-linux-slave42 manually (tag/source, maemo4 build). The desktop builders should be ok.
  • Keeping the same l10n-changesets from build1 yesterday afternoon.
  • Landed a release config update which contains changesets, build# bump, relbranch overrides.
  • Forced maemo4 to build on a VM:
diff -r faa11e08ea8d mozilla2/mobile_config.py
--- a/mozilla2/mobile_config.py Thu Jun 03 11:39:17 2010 -0700
+++ b/mozilla2/mobile_config.py Thu Jun 03 12:30:10 2010 -0700
@@ -15,7 +15,8 @@
 SBOX_HOME = '/scratchbox/users/cltbld/home/cltbld'
 
 MOBILE_SLAVES = {
-    'maemo4': config.SLAVES['linux'],
+#    'maemo4': config.SLAVES['linux'],
+    'maemo4': config.LINUX_VMS,
     'maemo5-gtk': config.SLAVES['linux'],
     'maemo5-qt': config.SLAVES['linux'],
     'linux-i686': config.SLAVES['linux'],
  • Reconfiged pm02, pmm 8011
  • Started the automation with:
buildbot sendchange --username=aki --master=localhost:9010 --branch=releases/mobile-1.1 "Fennec 1.1rc1 build2" doit

Tag

No problems. (yay!)

Er, noticed that this needed to be revision c6ad008b32ad:

  • killed the builds, clobbered mv-moz2-linux-ix-slave15 (source), moz2-linux-slave37 (maemo4 build), moz2-linux-slave35 (tag), stage (1.1rc1-candidates/build2) manually.
    • clobber == ssh in, mv /builds/slave/mobile_* /builds/slave/foo, rm -rf /builds/slave/foo (tag, source)
    • clobber == ssh in, mv /scratchbox/users/cltbld/home/cltbld/build/*release /builds/slave/foo, rm -rf /builds/slave/foo (maemo4 build)
  • landed release config change for posterity
  • tagged c6ad008b32ad manually
cd mobile-1.1
hg pull -u
hg up -r GECKO1925_20100602_RELBRANCH
hg tag -r c6ad008b32ad -f FENNEC_1_1rc1_BUILD2
hg tag -r c6ad008b32ad -f FENNEC_1_1rc1_RELEASE
hg outgoing | more
hg push
  • added dummy factory for tagging in release_mobile_master.py
  • reconfig
  • sendchange #2:
buildbot sendchange --username=aki --master=localhost:9010 --branch=releases/mobile-1.1 "Fennec 1.1rc1 build2 take2" doit

Source

No problems. s: moz2-linux-slave37

Build/Repack

  • Maemo4: No problems. [!]
  • Linux: No problems.
  • Mac: No problems.
  • Win32: No problems.

L10n waiting on slaves... we're down to 2 l10n slaves for linux on pm02. To speed things up, I:

        builders.append({
            'name': '%s_repack' % platform,
#            'slavenames': MOBILE_L10N_SLAVES['maemo4'],
            'slavenames': branchConfig['platforms']['linux']['slaves'],

and reconfiged.

Afterwards, the l10n repacks went well, except for fr which hung (seems like a lot of hg-related jobs hung on multiple machines at the same time). I killed that, force-built it but it died on get enUS source again (I think from a partial clone). Clobbered the dir on mv-moz2-linux-ix-slave16 and re-forced by going to master, going to builder "maemo4_repack" and setting:

#set name and 3 properties
locale fr
l10n_revision FENNEC_1_1rc1_RELEASE
en_revision FENNEC_1_1rc1_RELEASE

and that worked.

Partner Builds

  • First attempt timed out during repack step. Attempted "rebuild" from buildbot.
  • That worked! :)

Deb Repo Creation

Worked ! :)

I:

ssh cltbld@staging-mobile-master.build
cd /scratchbox/users/cltbld/home/cltbld
rm -rf mozharness repos logs configs     # This will get better I promise
hg clone http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness
mozharness/scripts/signdebs.py --logLevel debug --multiLog --configFile deb_repos/1.1_release.json

Then I looked in the logs/, noted that the warning, error, critical, and fatal logs were size 0, and did a quick check of the raw log to verify that I didn't miss any error regexes:

cat logs/signdebs_raw.log | egrep -v '(^Running |^Moving )' | more

I then took an n810, then:

Push Live

This is still accurate: https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Mobile#Copying_files_from_candidates.2F_to_releases.2F

I've also updated some of the rest of that page with brief 1.1 stuff.

Copy bits to releases/
Create new .install files
Create bouncer entries

Build 3

diff -r faa11e08ea8d mozilla2/mobile_config.py
--- a/mozilla2/mobile_config.py Thu Jun 03 11:39:17 2010 -0700
+++ b/mozilla2/mobile_config.py Thu Jun 03 12:30:10 2010 -0700
@@ -15,7 +15,8 @@
 SBOX_HOME = '/scratchbox/users/cltbld/home/cltbld'
 
 MOBILE_SLAVES = {
-    'maemo4': config.SLAVES['linux'],
+#    'maemo4': config.SLAVES['linux'],
+    'maemo4': config.LINUX_VMS,
     'maemo5-gtk': config.SLAVES['linux'],
     'maemo5-qt': config.SLAVES['linux'],
     'linux-i686': config.SLAVES['linux'],
  • Reconfiged pm02. Skipping pmm8011 since my mozharness script handles deb updates for releases.
  • Started the automation with:
buildbot sendchange --username=aki --master=localhost:9010 --branch=releases/mobile-1.1 "Fennec 1.1rc1 build3" doit

Tag

  • Died on el, which is a new locale added in Axel's l10n changeset landing. I knew this, but didn't realize I'd have to manually create the relbranch.
    • Brainstormed on how to fix this... by default create the relbranch if it doesn't exist on l10n repos only?
  • Manually created the relbranch in el:
hg clone http://hg.mozilla.org/releases/l10n-mozilla-1.9.2/el
cd el
# edit .hg/hgrc to allow for default-push
hg branch GECKO1925_20100602_RELBRANCH
hg commit
hg push
  • Edited l10n-changesets_mobile-1.1.json to remove all locales before el
  • Reconfiged; forced the tag builder to rebuild
  • Gah, clobbered mv-moz2-linux-ix-slave16, forced the tag builder to rebuild.
    • Tag factory needs more love =P
  • Backed out l10n-changesets_mobile-1.1.json local changes, added dummy tag_factory in release_mobile_master.py, reconfiged
  • Restarted automation with
buildbot sendchange --username=aki --master=localhost:9010 --branch=releases/mobile-1.1 "Fennec 1.1rc1 build3 post-tag" doit

Source

No problems. s: moz2-linux-slave37

Build/Repack

  • maemo4: No problems.
  • linux: No problems.
  • macosx: No problems.
  • win32: compilation error:
make[6]: *** No rule to make target `nsGB2312ToUnicodeV2.obj', needed by `ucvcn_s.lib'.  Stop.
  • However, a rebuild the next day worked. Need to investigate.
  • L10n repacks: No problems.

Partner Builds

  • Didn't trigger :P
    • Force build worked.

Deb Repo Creation

ssh cltbld@staging-mobile-master.build
cd /scratchbox/users/cltbld/home/cltbld
hg clone http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness
mozharness/scripts/signdebs.py --logLevel debug --multiLog --configFile deb_repos/1.1_release.json

Push Live

Copy bits to releases/

No problems

Create new .install files

Created new .install files. However, then I changed the firefox.install files back to chinook (only beta has fremantle/chinook split; release firefox.install should still point to chinook)

Create bouncer entries

Done, Firefox-Mobile-1.1rc1 and Firefox-Mobile-1.1rc1-N810

Re-done, Firefox-Mobile-1.1rc1 and Firefox-Mobile-1.1rc1-N900