QA/Desktop Firefox/Walkthroughs/Iteration Development

From MozillaWiki
Jump to: navigation, search

Iteration Development

The Firefox development team follows a two-week iteration cycle to land fixes on mozilla-central (aka Nightly).

Important Information

Meetings

  • Backlog Triage: Thursdays @ 10:00 Pacific
  • Sprint Planning: Tuesdays @ 12:30 Pacific

General Process

  1. The Desktop Management Team (Gavin, Chad and Madhava) review the Product Backlog and create a prioritized list of work for the upcoming iteration.
  2. Tuesday, a planning meeting is held for all teams to review and select bugs to be completed in the iteration. Anything that is not selected stays on the backlog for a future iteration.
  3. Tuesday, Desktop Project Managers distribute the current iteration assignments to the QA contract responsible for the current release cycle (Anthony, Tracy, or Juan).
  4. Wednesday, a QA triage session is held to determine which bugs need testing and to ensure those bugs are assigned to QA Contacts.
  5. As bugs selected for testing are resolved they are tested by the assigned QA Contacts.
  6. The following Tuesday, a status meeting is held for all teams to review the status of the selected work.
  7. At the end of the two-week iteration, all selected bugs should be resolved and tested as needed. Any work that is incomplete or not yet tested will be carried over to a future iteration.

Note: a demo meeting will occur the final week of the final iteration to present important features shipping in the current Nightly version.

v2 Proposal

Reducing the volume of [qa?] triage

QA believes there are some bugs which do not need to go through the [qa?] process. These bugs include work breakdowns, small UX/polish changes, and test-only changes. These bugs can be tagged [qa-] by PMs without first being tagged for QA review. QA will re-triage the [qa-] bugs once per iteration to make sure nothing is slipping through the cracks.

Dealing with near-migration landings

Some bugs will inevitably land too close to migration for QA to qualify in time. We will allow these bugs to ride the train to Aurora but they will carry over to the next iteration as a work item. If we find issues during that first Aurora iteration we will request a backout. That said, we would encourage only low-risk changes land in the last few days of the last iteration; high risk changes should wait for the next iteration to land. This is not just to ensure regressions don't sneak in but also reduces the likelihood of a risky/messy backout. As we get closer to migration, landing approvals should be looked at through the lense of Aurora uplift criteria. In other words, would Release Management approve your request for uplift if your patch was coming after migration? If the answer is no then we'd be inclined to ask you to hold off on landing the patch until after migration.

QA Commitment

QA will...

  • triage all bugs selected for work at the beginning of the iteration to identify work needing testing
  • ensure all bugs selected for testing are promptly assigned a QA Contact
  • ensure all bugs selected for testing are thoroughly tested upon resolution
  • not test any bugs deselected for testing unless a special request is made
  • not test any bug before it is RESOLVED unless a special request is made

QA Methodology

Important: this only applies to Firefox bugs at this time as the Platform and Services teams are not currently following this development model.

Whiteboard Tags

QA will use the following whiteboard tags to categorize bugs based on testing necessity:

  • [qa+]: needs testing before the iteration can be signed off, be sure these are assigned to a QA Contact (done - 3/20)
  • [qa-]: does not need testing before the iteration is signed off (breakdown bugs are [qa-] by default)
  • [qa?]: waiting for info from developers about how to test the fix, or whether they feel it needs verification.
  • [qa!]: testing has been completed and the bug is signed-off for release.

Canned Responses

  • [qa-]: "Please needinfo me if you'd like QA feedback on this bug."

Instructions for QA Leads

The QA Lead is responsible for ensuring all selected bugs are categorized and assigned for testing as appropriate (see Whiteboard Tags above), as well as ensuring all bugs are signed off prior to the end of the iteration. Any work missed or not completed in time will be carried over to the next iteration by the Product Management team.

1. Attend the Iteration Planning meeting on Tuesday so you know what work is being selected

2. Add a section to your test plan to track the selected bugs, you can use the bugzilla template. Here is the syntax for this live example: 

<bugzilla>
{
 "include_fields":"id,summary,status,qa_contact,whiteboard",
 "whiteboard":"s=it-31c-30a-29b.1"
}
</bugzilla>

3. Assign one of the whiteboard tags to each and every bug selected for the iteration

4. Assign a QA Contact to all [qa+] bugs (assign Florin Mezei if you are unsure)

5. Periodically recheck the list as new bugs may be added throughout the iteration

6. Review the list a couple days prior to the end of the iteration and ask for a status update on any unfinished bugs

Instructions for Testers

Testers are responsible for thoroughly testing all [qa+] bugs assigned to them before the iteration completes. The QA Lead will assigning bugs to individual testers on an as needed basis.

1. Before you begin testing, be sure to use the needinfo flag to ask the developer for clarification if you are unclear about what needs testing

2. Be sure to test that the code is working as indended and has not introduced regressions to pre-existing use cases

3. Document your testing in the bug including, tests performed, the platforms tested, and any issues you discovered

  • If you found no issues, mark the bug VERIFIED FIXED and change the whiteboard tag to [qa!]
  • If you found issues, mark the bug REOPENED and flag the developer for follow-up using the needinfo flag (see below)

When a FIXED bug has issues

  • Once a bug is resolved it is the responsibility of the QA Contact to ensure the bug is thoroughly tested.
  • If issues are found during testing they will be reported to the bug and that bug will be REOPENED.
  • It will be the Assignee's responsibility to determine if that work is in or out of scope.
  • If the issues are determined to be in-scope they will be addressed in that bug and retested by QA upon resolution.
  • If the issues are determined to be out-of-scope the bug will be marked VERIFIED FIXED and each of the issues will be reported by the QA Contact to dependency bugs.
  • Anything not completed before the iteration wraps up will be carried over to future iterations.
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Desktop_Firefox/Walkthroughs/Iteration_Development&oldid=974418"