Features

Every version of Firefox comes with new features. For the most part these are tracked in Bugzilla using the feature keyword. It is the responsibility of QA to ensure these bugs are thoroughly tested so they can be shipped in time, or disabled if not ready. This should also be extended to include bugs with [meta] in their summary.

QA Commitment

QA will...

• triage and track all bugs properly tagged with the feature keyword
• ensure feature bugs are thoroughly tested prior to release
• ensure issues concerning possible delayed uplift of a feature are escalated in a timely manner
• NOT rush testing to meet ship targets if the feature is "crash landed" or if the feature is brought to our attention late in the cycle
• NOT take responsibility for the quality of features that are not properly tracked in Bugzilla

QA Methodology

QA Contact

• will be assigned for every feature bug
• will be responsible for testing and reporting subsequent issues related to the feature
• will be responsible for signing off the feature at each migration stage

Flags

• qe-verify+: a feature bug that will be tested and signed-off before release - This is the equivalent of the deprecated whiteboard tag [qa+]
• qe-verify-: a feature bug that will not receive any special QA attention before release unless explicitly requested - This is the equivalent of the deprecated whiteboard tag [qa-]
• qe-verify?: a feature bug that needs QA/Dev review to determine if it needs testing or not - This is the equivalent of the deprecated whiteboard tag [qa?]
• [qa!] is a whiteboard tag that was used to mark a feature bug that has had all requisite testing completed and is signed off for release - This is no longer used an has no equivalent flag, as it was determined that we can use the status flags to determine when verification is complete

Instructions for QA Leads

The QA Lead is responsible for ensuring all tracked feature bugs are assigned to a relevant QA Contact.

1. Add a section to your test plan to track feature bugs using this example syntax (live example):

Front-end:

<bugzilla>
{
 "include_fields":"id,summary,status,qa_contact,whiteboard",
 "keywords":"feature",
 "target_milestone":"Firefox 31"
}
</bugzilla>

Back-end:

<bugzilla>
{
 "include_fields":"id,summary,status,qa_contact,whiteboard",
 "keywords":"feature",
 "target_milestone":"mozilla31"
}
</bugzilla>

2. Categorize each of the bugs using one of the above whiteboard tags

3. Assign each of the [qa+] tagged bugs to a QA Contact (Florin Mezei if you are unsure)

4. Periodically review the list to make sure all bugs have an accurate QA status

5. Contact Release Management immediately if a QA Contact identifies a feature with possible blockers

Instructions for Testers

QA Testers are responsible for testing and signing off all feature bugs they are assigned. This includes reporting follow-up issues and escalating potential blockers to releasing the feature in the target milestone.

1. Draft a test plan and work with the developer to ensure all use cases are covered by manual and automated tests

2. Try to confirm bugs as they are reported and identify regression windows where appropriate

3. Try to verify fixes as they are landed on mozilla-central, uplifted to mozilla-aurora, or uplifted to mozilla-beta

4. Ensure any serious bugs are reported and escalated to the developer and QA lead

5. Mark the feature tracking bug as VERIFIED FIXED once the initial implementation has been signed off for release

6. Send a sign-off email to the developer(s) and the QA lead prior to each branch migration

7. Contact the QA Lead immediately if you think a feature needs to be disabled

Sign-off Criteria

• Uplift to Aurora - feature use cases have been verified with no serious stability, performance, or regression concerns
• Uplift to Beta - edge-cases have been verified with no serious stability, performance, or regression concerns
• Uplift to Release - feature polish has been verified with no serious stability, performance, or regression concers