QA/Templates/Firefox Desktop Testplan
- TEST PLAN SHOULD BE CREATED AT wiki.mozilla.org/Releases/Firefox_VERSION/Test_Plan
« Firefox PREVIOUS | Firefox CURRENT | Firefox NEXT »
Contents
Firefox VERSION Test Plan
Summary
Milestone | [LINK_TO_FLIGHT_TRACKING_PAGE Firefox VERSION] |
Status | Currently in BRANCH -- merging to NEXT_BRANCH on DATE_OF_MERGE |
Lead | [MAILTO_LINK_FOR_LEAD] (irc: NICK) |
Peer | [MAILTO_LINK_FOR_PEER] (irc: NICK) |
Latest Build: [ ftp], [ http], [ web]
The following is the test plan for Firefox VERSION from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox VERSION. After the release this document will be used as an archive of what was done to validate this release.
If you have some free time, please pick a task below and get in touch with one of the leads.
Details
The following is a description of the tasks which need to be completed throughout this release.
- Daily Tasks
- {{|}} [tester] Automation: Check dashboards for new failures
- {{|}} [tester] Bug Triage: Check Bugzilla for regressions
- {{|}} [tester] Features: Test features and triage feature bugs targeted for this releases
- {{|}} [tester] Input: Triage Input.MO for prevalent issues and feature requests
- One-time-only Tasks
- {{|}} [tester] Nightly Merge: Sign-off initial merge to Nightly
- {{|}} [tester] Aurora Merge: Sign-off merge from Nightly to Aurora
- {{|}} [tester] Beta Merge: Sign-off merge from Aurora to Beta
- {{|}} [tester] Release: Sign-off final release
Automation
BFTs
Triage the [ Mozmill dashboard] for test failures and debug for potential Firefox regressions.
- Lead: Vlad Maniac (Softvision)
- Peer: Alex Lakatos (Softvision)
- Failure Status: NEW (0) | ASSIGNED (0) | FIXED (0) | REGRESSIONS (0)
- Tasks:
- Check the Mozmill dashboard for failing tests for the day
- Cross reference any failure to the list of NEW and ASSIGNED bugs
- If the bug is NEW, assign it to yourself and try to fix it
- If the bug is ASSIGNED and there has been no activity on the bug in over a week, offer your assistance to fix the bug
- If there is NO bug on file, file it using this template
- If you need assistance, ask ashughes in the #mozmill channel
Endurance
Triage and report the performance numbers from the daily Endurance tests.
- Endurance reports
- Spreadsheet for tracking results
- NOTE: Report any spikes in performance degradation to the QA release driver
- Issues Found
Bug Triage
Check UNCONFIRMED, NEW, FIXED, REOPEN, and tracking/status bugs
- Lead: Vlad Maniac (Softvision)
- Peer: Alex Lakatos (Softvision)
- Bugs:: UNCONFIRMED, FIXED, TRACKING/STATUS
- Tracking: UNCONFIRMED Spreadsheet, FIXED Spreadsheet
- Important Bugs
- [STATUS] bug - Bug summary
- Notes
Features
Track features which have landed in Firefox VERSION and are ready to be tested.
FEATURE
SUMMARY OF THE FEATURE, GOALS, ETC
- Leads: NAME (ROLE)
- QA Owner: NAME
- Status: STATUS
- Links: TEST PLAN, TRACKING BUG, ETC
- Notes:
Input
The following are the details of daily triage conducted on http://input.mozilla.org.
- Tasks
- Go to http://input.mozilla.org
- Filter the results by Firefox version
- Any feedback which is spiking should listed below and escalated to the QA release driver
- Popular Feedback
Release
The following tasks need to be completed to sign-off the Beta merge.
- {{|}} [tester] Automated tests: details
- {{|}} [tester] Manual tests: details
- {{|}} [tester] Feature spotchecks: details
- {{|}} [tester] Bug fix verifications: details
- {{|}} [tester] Updates: details
Automated Tests
The following are the detailed results of the automated BFT test runs.
- {{|}} Windows 2000: [ functional] | [ restart]
- {{|}} Windows XP: [ functional] | [ restart]
- {{|}} Windows Vista: [ functional] | [ restart]
- {{|}} Windows 7: [ functional] | [ restart]
- {{|}} Windows 7 64-bit: [ functional] | [ restart]
- {{|}} Mac OS X 10.6: [ functional] | [ restart]
- {{|}} Ubuntu 11.04: [ functional] | [ restart]
- {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
- Reports: [ Litmus], [ Mozmill]
- Issues Found
Manual Tests
The following are the detailed results of the manual smoketests.
- {{|}} [tester] Windows 2000: status
- {{|}} [tester] Windows XP: status
- {{|}} [tester] Windows Vista: status
- {{|}} [tester] Windows 7: status
- {{|}} [tester] Windows 7 64-bit: status
- {{|}} [tester] Mac OS X 10.6: status
- {{|}} [tester] Ubuntu 11.04: status
- {{|}} [tester] Ubuntu 11.04 32-bit: status
- Litmus [ testrun], [ report]
- Firefox VERSION [ builds]
- Issues Found
- Exploratory Testing
Conduct exploratory tests on FOCUS AREA
- {{|}} [tester] Windows XP: status
- {{|}} [tester] Windows 7: status
- {{|}} [tester] Mac OS X 10.6: status
- {{|}} [tester] Ubuntu 11.04: status
- Issues Found
Features
The following are the details about spotchecks performed on landed features
- {{|}} [tester] FEATURE: status
- Issues Found
Bug Verifications
The following are the details of bug verifications for Firefox VERSION
- Bugs: [ tracking-firefoxN+]
- Results: [ VERIFIED], [ REOPENED]
- Issues Found
Updates
The following are details of update verifications for Firefox VERSION
Betatest: report
- {{|}} Windows XP: status
- {{|}} Windows Vista: status
- {{|}} Windows 7: status
- {{|}} Windows 7 64-bit: status
- {{|}} Mac OS X 10.6: status
- {{|}} Mac OS X 10.7: status
- {{|}} Ubuntu 11.04: status
- {{|}} Ubuntu 11.04 64-bit: status
Beta: report
- {{|}} Windows XP: status
- {{|}} Windows Vista: status
- {{|}} Windows 7: status
- {{|}} Windows 7 64-bit: status
- {{|}} Mac OS X 10.6: status
- {{|}} Mac OS X 10.7: status
- {{|}} Ubuntu 11.04: status
- {{|}} Ubuntu 11.04 64-bit: status
Releasetest: report
- {{|}} Windows XP: status
- {{|}} Windows Vista: status
- {{|}} Windows 7: status
- {{|}} Windows 7 64-bit: status
- {{|}} Mac OS X 10.6: status
- {{|}} Mac OS X 10.7: status
- {{|}} Ubuntu 11.04: status
- {{|}} Ubuntu 11.04 64-bit: status
Release: report
- {{|}} Windows XP: status
- {{|}} Windows Vista: status
- {{|}} Windows 7: status
- {{|}} Windows 7 64-bit: status
- {{|}} Mac OS X 10.6: status
- {{|}} Mac OS X 10.7: status
- {{|}} Ubuntu 11.04: status
- {{|}} Ubuntu 11.04 64-bit: status
- Issues Found
Beta Merge
The following tasks need to be completed to sign-off the Beta merge.
- {{|}} [tester] Automated Tests: details
- {{|}} [tester] Manual Tests: details
- {{|}} [tester] Features: details
- {{|}} [tester] Bug fix verifications: details
- {{|}} [tester] Updates: details
Automated Tests
The following are the detailed results of the automated BFT test runs.
- {{|}} Windows 2000: [ functional] | [ restart]
- {{|}} Windows XP: [ functional] | [ restart]
- {{|}} Windows Vista: [ functional] | [ restart]
- {{|}} Windows 7: [ functional] | [ restart]
- {{|}} Windows 7 64-bit: [ functional] | [ restart]
- {{|}} Mac OS X 10.6: [ functional] | [ restart]
- {{|}} Ubuntu 11.04: [ functional] | [ restart]
- {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
- Reports: [ Litmus], [ Mozmill]
- Issues Found
Manual Tests
The following are the detailed results of the manual smoketests.
- {{|}} [tester] Windows 2000: status
- {{|}} [tester] Windows XP: status
- {{|}} [tester] Windows Vista: status
- {{|}} [tester] Windows 7: status
- {{|}} [tester] Windows 7 64-bit: status
- {{|}} [tester] Mac OS X 10.6: status
- {{|}} [tester] Ubuntu 11.04: status
- {{|}} [tester] Ubuntu 11.04 32-bit: status
- Litmus [ testrun], [ report]
- Firefox VERSION [ builds]
- Issues Found
- Exploratory Testing
Conduct exploratory tests on FOCUS AREA
- {{|}} [tester] Windows XP: status
- {{|}} [tester] Windows 7: status
- {{|}} [tester] Mac OS X 10.6: status
- {{|}} [tester] Ubuntu 11.04: status
- Issues Found
Features
The following are the details about spotchecks performed on landed features
- {{|}} [tester] FEATURE: status
- Issues Found
Bug Verifications
The following are the details of bug verifications for Firefox VERSION
- Bugs: [ tracking-firefoxN+]
- Results: [ VERIFIED], [ REOPENED]
- Issues Found
Updates
The following are details of update verifications for Firefox VERSION merge to Beta
- Betatest
- {{|}} [tester] Windows XP: status
- Source:
- Target:
- {{|}} [tester] Windows 7: status
- Source:
- Target:
- {{|}} [tester] Mac OS X 10.6: status
- Source:
- Target:
- {{|}} [tester] Ubuntu 11.04: status
- Source:
- Target:
- Beta
- {{|}} [tester] Windows XP: status
- Source:
- Target:
- {{|}} [tester] Windows 7: status
- Source:
- Target:
- {{|}} [tester] Mac OS X 10.6: status
- Source:
- Target:
- {{|}} [tester] Ubuntu 11.04: status
- Source:
- Target:
- Issues Found
Aurora Merge
The following tasks need to be completed to sign-off the Aurora merge.
- {{|}} [tester] Automated Tests: details
- {{|}} [tester] Manual Tests: details
- {{|}} [tester] Features: details
- {{|}} [tester] Bug fix verifications: details
- {{|}} [tester] Updates: details
Automated Tests
The following are the detailed results of the automated BFT test runs.
- {{|}} Windows 2000: [ functional] | [ restart]
- {{|}} Windows XP: [ functional] | [ restart]
- {{|}} Windows Vista: [ functional] | [ restart]
- {{|}} Windows 7: [ functional] | [ restart]
- {{|}} Windows 7 64-bit: [ functional] | [ restart]
- {{|}} Mac OS X 10.6: [ functional] | [ restart]
- {{|}} Ubuntu 11.04: [ functional] | [ restart]
- {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
- Issues Found
Manual Tests
The following are the detailed results of the manual smoketests.
- {{|}} [tester] Windows 2000: result
- {{|}} [tester] Windows XP: result
- {{|}} [tester] Windows Vista: result
- {{|}} [tester] Windows 7: result
- {{|}} [tester] Windows 7 64-bit: result
- {{|}} [tester] Mac OS X 10.6: result
- {{|}} [tester] Ubuntu 11.04: result
- {{|}} [tester] Ubuntu 11.04 32-bit: result
- Litmus [ testrun], [ report]
- Firefox VERSION [ builds]
- Issues Found
Features
The following are the details about spotchecks performed on landed features
- {{|}} [tester] FEATURE: status
- Issues Found
Bug Verifications
The following are the details of bug verifications for Firefox VERSION
- Bugs: [ tracking-firefoxN+]
- Results: [ VERIFIED], [ REOPENED]
- Issues Found
Updates
The following are details of update verifications for Firefox VERSION merge to Aurora
- Auroratest
- {{|}} [tester] Windows XP: result
- Source:
- Target:
- {{|}} [tester] Windows 7: result
- Source:
- Target:
- {{|}} [tester] Mac OS X 10.6: result
- Source:
- Target:
- {{|}} [tester] Ubuntu 11.04: result
- Source:
- Target:
- Aurora
- {{|}} [tester] Windows XP: result
- Source:
- Target:
- {{|}} [tester] Windows 7: result
- Source:
- Target:
- {{|}} [tester] Mac OS X 10.6: result
- Source:
- Target:
- {{|}} [tester] Ubuntu 11.04: result
- Source:
- Target:
- Issues Found
Nightly Merge
The following tasks need to be completed to sign-off the Nightly merge.
- {{|}} [tester] Automated Tests: details
- {{|}} [tester] Manual Tests: details
- {{|}} [tester] Features: details
- {{|}} [tester] Bug fix verifications: details
- {{|}} [tester] Updates: details
Automated Tests
The following are the detailed results of the automated BFT test runs.
- {{|}} Windows 2000: [ functional] | [ restart]
- {{|}} Windows XP: [ functional] | [ restart]
- {{|}} Windows Vista: [ functional] | [ restart]
- {{|}} Windows 7: [ functional] | [ restart]
- {{|}} Windows 7 64-bit: [ functional] | [ restart]
- {{|}} Mac OS X 10.6: [ functional] | [ restart]
- {{|}} Ubuntu 11.04: [ functional] | [ restart]
- {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
- Issues Found
Manual Tests
The following are the detailed results of the manual BFTs.
- {{|}} [tester] Windows 2000: result
- {{|}} [tester] Windows XP: result
- {{|}} [tester] Windows Vista: result
- {{|}} [tester] Windows 7: result
- {{|}} [tester] Windows 7 64-bit: result
- {{|}} [tester] Mac OS X 10.6: result
- {{|}} [tester] Ubuntu 11.04: result
- {{|}} [tester] Ubuntu 11.04 32-bit: result
- Litmus [ testrun], [ report]
- Firefox VERSION [ builds]
- Issues Found
Features
The following are the details about spotchecks performed on landed features
- {{|}} [tester] FEATURE: status
- Issues Found
Bug Verifications
The following are the details of bug verifications for Firefox VERSION
- Bugs: [ tracking-firefoxN+]
- Results: [ VERIFIED], [ REOPENED]
- Issues Found
Updates
The following are details of update verifications for Firefox VERSION merge to Nightly
- Nightly
- {{|}} [tester] Windows XP: result
- Source:
- Target:
- {{|}} [tester] Windows 7: result
- Source:
- Target:
- {{|}} [tester] Mac OS X 10.6: result
- Source:
- Target:
- {{|}} [tester] Ubuntu 11.04: result
- Source:
- Target:
- Issues Found