Firefox/Channels/Postmortem/69

From MozillaWiki
Jump to: navigation, search

Notes for Firefox 69 release post mortem

10am PT September 24, 2019 Zoom Meeting ID: 266 532 573 IRC: #release-drivers / slack: #release-coordination

Attendees

  • marcia, RyanVM, Thomas, lizzard, Ritu, Aryx, Tom, dveditz,Tania, nhi

Stats

  • Nightly: May 20 to July 8 (7 weeks)
  • Beta: July 8 to September 2 (8 weeks)
  • Uplifts to Beta: 359 (362 in 68)
  • Uplifts to RC: 12 (22 in 68)
  • Fixed in 69: patches from 4159 bugs (5353 in 68)
    • 795 fixed regressions - 356 new, 439 carry-over
  • 433 unfixed carry-over regressions
  • QA Stats
    • QA accepted total 22 desktop features after feature prioritization
    • 10 features moved to Backlog/future releases (some of them were pushed to make room for Relationship KPI work)
    • PI Request deadline was May 31, 2 features missed the deadline, 1 of them was related to Relationship KPI
    • Technical documentation deadline was June 14, 5 (including one Relationship KPI feature, 2 are Security features) features missed the deadline
    • Code readiness date was June 28, 1 feature (Relationship KPI) landed late
    • QA Bugs by feature: https://docs.google.com/spreadsheets/d/1mgDYAFbHvpbcvo9S_XDhqK-S2n_MsP5a6uAJWc5J_9I/edit#gid=0

What went well

  • Only one RC build needed
  • Relationship scoping work didn't cause any major disruptions to the release
  • The deprioritization list created to help accommodate the Relationship KPI changes was very useful
  • The cycle went smooth, no issues occured. inspite of late addition to the Release, Relationship KPI work went well
  • Talking with developers was easy and accessible. Sign-offs were sent on time or even earlier.
  • New people (Arg & RO) fit well with the rest of the team
  • ETP shipped on by default to all users with minimal issues
    • Issues with Facebook Instant Games fixed via Remote Settings update (bug 1580416)
  • No unexpected major regressing issues got reported after the v69 rollout on our support channels
  • [Read-only] Add-Ons QA - very explicit test plan provided from the start. In most cases, developer providing timely and useful input.
    • The information provided with the PI request was sufficient and the lead developer was of great help by promptly answering any questions which have arisen during testing. All in all, the whole process has transitioned from start to finish very smoothly.
  • [Read-Only] Experiments QA - Communication with the eng teams was good, luckily we've been collaborating for quite a while (Pocket & User Journey).
    • The developers are very prompt with replies to emails/questions. They provided us with add-ons and detailed json blobs for testing.

Challenges

  • Bug 1567224: WNP locales not finalized until Monday of RC week
    • Late completion of es-ES prevented the list from being finalized sooner
  • 60.9esr delays due to patch rebase
    • IndexedDB security fix (bug 1501152) needed fixing on all branches
    • Issues with how the bug was triaged and fixed (Fixed on m-c in late July)
    • Limited resources and confusion over ESR60 status slowed things down
  • Bug 1567600: Addons Manager a11y regressions
    • Regressed by about:addons rewrite landed in 68
    • Patches weren't ready in time for RC build, had to land in 69.0.1 after going through Exception Request process
  • Bug 1578136: Secure Proxy affected by late-reported crash bug during RC week
    • Uplifted to 69.0.1
    • Had similar late-reported issues in Fx68 which led to uplifts in 68.0.1
  • Bug 1567235: Apple Notarization requirements caused long gtb delays earlier in Beta cycle
    • Things seems to be running smoothly now since mitigations landed
  • Read-Only [Experiments QA] Normandy features had no documentation when the PIs were filed, and they were filed late during the cycle
  • Read-Only {Add-ons QA] Time spent researching bugs that were "qe-" but did not have the flag set. Time spent researching bugs that have: not clear enough repro steps/ no repro, not enough context, expected results are a puzzle to put together from various comments (ex: 1452713).
Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox/Channels/Postmortem/69&oldid=1218243"