Firefox/Channels/Postmortem/38.0.5
From MozillaWiki
Firefox 38.0.5 Post-mortem
Summary: we did a 38.0.5 release to match a marketing / PR campaign. To make sure that builds are well tested, we used the beta channel (but building from the mozilla-release repo). We also pushed a what's new for 40.
Many of the points have been discussed during the previous Post Mortem meeting: https://wiki.mozilla.org/Firefox/Channels/Postmortem/38
Repeating:
- (sylvestre) major release out of the schedule should be avoided at all cost
Infrastructure that broke because of the extra release numbering
- updates weren't available for Firefox 38.0.5 build3 on the beta-cdntest channel.. . it turns out that 38.0.5 has broken yet another assumption in our release automation which caused it not to prepare updates on the beta channel and test channels. (5/22)
- This delayed the build of 39 beta 1
- (sylvestre) Ship-it was giving beta builds for the release partials (because we built beta from the release repo)
- https://github.com/mozilla/ship-it/commit/6f809e28e1b77bd30f219cad5cdc25f8a26fa428
- (sylvestre) What's new on dev edition / aurora didn't work (never before attempted - new feature)
- https://bugzilla.mozilla.org/show_bug.cgi?id=1143875
- Can we rethink this page? Would be good to push it live whenever we want not just on release
- This would probably simplify the QE job
- (sylvestre) Release notes didn't go live (bug in nucleus)
- (sylvestre) funnelcake
- buildbot (?) was expecting win64 builds for past versions, but they didn't exist, so the build failed (Not sure that a bug was filed here)
- (KaiRo) Socorro crash-stats needed a specific exception to recognize that a 3-part version can be shipped to the beta channel as an RC. Took us 5 days of delay until it was fixed until we did get data from those RCs. See bug 1156961.
- (KaiRo) Because we are doing intermediate releases like this and trying to ship features so fast, the quality of the product suffers, in this case esp. of 39, which has a shorter cycle of beta testing and we know we need multiple weeks to react to data and get data from reactions again.
- Pocket integration quality
- rushing
- not enough time for testing, user testing and debug
- (lmandel) AMD crash required that we skip a 39 beta release. We really need an answer for the AMD crash. (But we already know this.)
- we had many occurrences of this bug (38.0 b2, b5, b8 & b9 affected)
- (lmandel) Delay of 39 beta resulted in external press inquiries. Just so that people are aware that those outside of Mozilla notice these changes.
- (elan) We did it! Even with all of the items listed above! Thank you!
- (sylvestre) Extra hours, extra stress
- Impacted Releng, QE, developer, project management, web team, relman, etc
- A lot of people worked during week end and late
(Ritu) Reduced the 39 Beta cycle by over 2 weeks which is a huge impact on release stability and quality!
Post Mortem on Friday as well. Survey: https://docs.google.com/a/mozilla.com/forms/d/1_02MEAoKeXdMwcLtiZfM3ii2tbnAB3uR1ji51DjwMiU/viewform?c=0&w=1
Friday @ 1PM in Program Management
