Releases:Release Post Mortem:2016-02-03
From MozillaWiki
Meeting Details
- 3:30pm ET
- Vidyo - Release Engineering room
- Dial-in:
- 650-903-0800 or 650-215-1282 x92 Conf# 98225 (US/INTL)
- 1-800-707-2533 (pin 369) Conf# 98225 (US)
- #releaseduty on irc.mozilla.org
- https://trello.com/b/MXHaVRcP/release-promotion-meeting
« previous week |
index |
next week »
< most recent |
upcoming >
Contents
Release Duty
- FF 45 cycle: mtabara
Misc
Shipped
Firefox/Fennec_44.0 (jlund/rail/mtabara)
- Fennec 44.0 build2 went smoothly
- pushed desktop Firefox 44.0 build1 to beta channel.
- Some intermittent failures came up on buil1:
- repack_10/10 complete for Firefox 44.0 build1 on linux, re-triggered
- Firefox 44.0 build1: failed at repack_7/10 on win32, retry
- merge mozconfig issue with mozilla-release and tools whitelist not being updated
- resolved: Bug 1233262 - add ENABLE_RELEASE_PROMOTION variable to sanity whitelist for mozilla-release
- There's a follow-up build2. It might be a 43 m-r based release with selectively uplifted sec fixes, not yet decided - awaiting more data on crash-rates IIUC
- Some intermittent failures came up on buil2:
- firefox_source failed for Firefox 44.0 build2 - failed to checkout mozilla-release. jlund suspected this is related to scl3 network issues mentioned in #moc, retried
- Firefox 44.0 build2: failed at update_verify_beta_2/6 on linux64 - gtk3 errors that can be ignored
- all update verify tasks in beta for linux/linux64/macosx64 failed mostly because of binary files found in diff - could be that we built off of m-b. Tracking bug for this is bug 1241951
- desktop ff - no antivirus? no email. not in pending/recent/running.
- there is reason to believe antivirus was never ran because of infra issues/downtime bug 1241334 that overlapped with the timing of 44.0 gtb
- manually re-triggered to force build outside of the release graph
- initially manual re-triggering failed for a downloading issue, retriggered that once again
- status: retriggered worked. blame missing antivirus on https://bugzil.la/1241334
- Firefox 44.0 build3. usual update verify gtk errors but flawless victory
- bhearsum manually synced the SDKs to the releases dir due to bug 1243740 with local patch applied:
- Firefox 38.6.0esr build1 is too on fire and going well in the pipeline
- shipped. flawless victory
Firefox/Fennec_45.0b1 (jlund/rail/callek/nick)
- TODO sum-up, where do we stand with the GTK3 known issue?
- gtk3 not supported via buildbot based builds
- solution: Bug 1242641 - GTK+3 still not working for buildbot builds on beta
- bhearsum manually synced the SDKs to the releases dir due to bug 1243740 with the local patch applied:
- nthomas manually ran checksums/push-to-mirrors because source builders failed
- jlund forced antivirus for the same ^ reason
- Bug 1243448 - beta release source builder failing to run configure
- status: resolved for beta1. needs follow up
- nthomas re-ran checksums (again) because we fixed source builders after above re-triggers and so source artifacts were missing. also missing, SUM and KEY bits which he manually pushed
- [Bug 1243873] New: KEY file missing for releases
Firefox/Fennec_45.0b2 (jlund/rail/callek/nick/mtabara)
- gtk3 known issue errors in update verify steps
- flawless victory
Ongoing
Thunderbird_45.0b1 (jlund/rail/callek/nick/mtabara)
- TODO: awaiting antivirus check and it's all good to release
- we had two win32 repacks failing
- failed at repack_6/10 on win32 - retriggered, intermittent timeout
- failed at repack_2/10 on win32 - retriggered
- retriggered upon 'da' locale failed while submitting to balrog, specifically around the make_incremental_update.sh script
- retriggered upon loosing slave instance
- retriggered upon timeout
Merge week
- migration went smoothish
- bug 1242892 - [Fennec] Wrong branding for Aurora 46.0a2 on API11 builds
- resolved