QA/Firefox3.5/TestPlan/Beta2
Contents
Firefox 3.1 Beta2 Test Results
The covered testing for this release will focus on smoketests, ffts, l10n, top extensions, top plugins, updates, and spotchecks.
Related Pages
Contact Info
PM: Mike Beltzner (beltzner)
Build: John O'Duinn (joduinn)
QA: Tony / others
Schedule
- Code Freeze
- 11/4
- String Freeze
- 11/10
- Builds hand to QA
- 11/25
- QA test signoff
- 12/5
- See Test Coverage below
Build Location and IDs
Download:
Builds:
- Windows XP: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2
- Mac: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2
- Linux: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b2) Gecko/20081201061202 Firefox/3.1b2
Test Coverage
- Areas Covered:
- Smoketests
- BFTs
- L10n
- Updates
- Top Extensions
- Accessibility Spot check
- Spotchecks & Webpages
- Bug Fixes
- Areas Not Covered:
- FFTs
- Topsites (covered in nightly automation)
- Top Themes
- Top Plugins
Test Results
| Types of Tests | Windows XP | Mac OSX | Linux |
|---|---|---|---|
| Smoketests | marcia - PASS | Tracy - PASS | Joel - PASS |
| BFTs | marcia/Tony - PASS | tracy - PASS | Joel - PASS |
| L10n | Tony - PASS | Tony - PASS | Tony - PASS |
| Top Extensions | Murali - PASS | Murali - PASS | Murali - MIXED |
| Accessibility | Marco - PASS | N/A | Marco - PASS |
| Updates | Murali Betatest - PASS releasetest - PASS Beta - PASS |
Tony Betatest - PASS releasetest - PASS Beta - PASS |
Tracy/Murali Betatest - PASS releasetest - PASS Beta - PASS |
| Live Spotchecks & Webpages | Tony - PASS | Tony - PASS | Murali - PASS |
Smoketest Testing
Lite verification tests (or smoketests) will be first ran when the builds are ready. Tests are available on litmus under the Firefox 3.1 test suite. In any case the Smoketests result, we will raise blockers asap before proceeding with the bfts.
Please record your detailed results in the section below.
BFT / FFT Testing
BFTs are a more comprehensive set of litmus tests that cover the build testing after smoketests result. Tests are available on litmus under the Firefox 3.1 test suite.
FFTs are a full comprehensive set of litmus tests that extend coverage in areas that BFTs dont often get to. These are run at least once during the major milestone timeline.
If you are running a BFT or FFT, Please record your results in the Detailed Results section
Updates Testing
We will be doing updates of all Alpha users to Beta2. We will be checking complete updates, and resultover scenarios.
Will be available on betatest, releasetest, and beta channels.
Testcases and Results are tracked here.
L10N
Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to L10N Requirements for more specifics.
Testcases and Results are tracked here.
Top Extensions Testing
Select a few top extensions from AMO and perform the following testcases. See Fx3 Compatibility Addons List Here.
Also, this is the top 50 most popular addons voted by mozdev.
Testcases:
- Enabling extensions and confirming web compatibility
- Disabling or Uninstalling extensions and confirming proper messaging and behavior
- Load about 10+ extensions, and fire them up all at once, verifying functionality and performance
- Updating old extension versions to the latest if applicable
- Switch profiles with different extensions, and confirm they do not get carried over.
Testcases and Results are tracked here.
Accessibility
Ensure UI pieces such as awesomebar, ctrl-tab, private browsing, etc.. are working correctly and follows a11y standards.
Live Spotchecks & Webpages
1) Verify the following webpages that gets pushed to staging and then to the live sites. Look for content messaging, spelling mistakes, broken links & imnages, etc..
- Firstrun page
- Whatsnew Page
- Releasenotes page
2) Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:
- Check for signed installer and applications (windows only)
- Firstrun page, whatsnew page, Home page, Search plugin
- Install, uninstall
- General spotchecks and browser functionality
- Confirm Build IDs
Fixed Bugs Verifications
The following bugs are fixed for Beta 2 and need verifications:
- Platform 1.9.1 bugs: http://tinyurl.com/583w29
- Firefox 3.1 bugs:http://tinyurl.com/6dr9uu
Testday
A Beta2 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.
Detailed Results
Smoketests Results
Windows XP [marcia]
- PASS
- Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2) Gecko/20081125 Firefox/3.1b2
- Tear away tab feature does not seem as smooth as when running on Mac. Win XP machine does not have a lot of memory so this may be the cause.
Mac [tracy]
- 11/15 - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.1b2) Gecko/20081125 Firefox/3.1b2 - PASS
- Results in Litmus
- Didn't run install ext or theme test cases. none compatible, not even nightly tester tools.
Linux [jmaher]
- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b2) Gecko/20081125 Firefox/3.1b2 - PASS
BFT Results
Windows XP [marcia/Tony]
- Filed bug 467601 - long bookmark descriptions will hide tagging icon and text
Mac [tracy]
- Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 - PASS
