QA/Firefox3.5/TestPlan/Beta4
Contents
Firefox 3.5 beta4 Test Results
The covered testing for this release will focus on smoketests, bfts, l10n, top plugins, top financial sites, updates, and spotchecks.
Related Pages
Project contacts
PM: Mike Beltzner (beltzner)
Build: Nick Thomas (nthomas)
QA: Tony Chung (tchung)
Schedule
- Code Freeze
- 4/22?
- Builds hand to QA
- 4/23?
- QA test signoff
- 4/27?
- See Test Coverage below
Build Location and IDs
Download:
Builds:
- Mac: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4
- Linux: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4
Test Coverage
- Areas Covered:
- Smoketests
- BFTs
- L10n
- Updates
- Spotchecks & Webpages
- Bug Fixes
- Areas Not Covered:
- FFTs
- Topsites (covered in nightly automation)
- Top Extensions
- Top Themes
- Top Plugins
- Top Financial Sites
- Accessibility
Test Results
Types of Tests | Windows XP | Mac OSX | Linux |
---|---|---|---|
Smoketests | Henrik - PASS (unsigned) Timr - PASS (signed) |
Aakash/Timr - PASS | Tracy - PASS Though randomly crashed a couple times. |
BFTs | Henrik/Murali - PASS | Marcia/Tchung - PASS | Aakashd/Tracy - PASS |
L10n | Tchung - PASS | Tchung - PASS | Tchung - PASS |
Updates | Tracy Betatest - PASS releasetest - PASS Beta - result |
Henrik/Murali Betatest - PASS releasetest - PASS Beta - PASS |
Aakashd/tester? Betatest - PASS releasetest - PASS Beta - PASS |
Additional Focused Testing | Fx-team, QA team - PASS | Fx-team, QA team - PASS | Fx-team, QA team - PASS |
Topsite Testing | Tomcat - PASS | Tomcat - PASS | Tomcat - PASS |
Staging/Live Webpages & Spotchecks & Webpages | <web tester>, henrik - result | <web tester>, marcia - result | <web tester>, Aakashd - PASS |
Smoketest Testing
Lite verification tests (or smoketests) will be first ran when the builds are ready. Tests are available on litmus under the Firefox 3.1 test suite. In any case the Smoketests result, we will raise blockers asap before proceeding with the bfts.
Please record your detailed results in the section below.
BFT / FFT Testing
BFTs are a more comprehensive set of litmus tests that cover the build testing after smoketests result. Tests are available on litmus under the Firefox 3.1 test suite.
FFTs are a full comprehensive set of litmus tests that extend coverage in areas that BFTs dont often get to. These are run at least once during the major milestone timeline.
If you are running a BFT or FFT, Please record your detailed results in the section below.
Updates Testing
We will be doing updates of all Alpha/Beta users to beta4. We will be checking complete updates, and failover scenarios.
Will be available on betatest, releasetest, and beta channels.
Testcases and Results are tracked here.
L10N
Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to Fx3.1 L10N Requirements for more specifics.
Testcases and Results are tracked here.
Top Site Testing
- Tests on all platforms pass without a crash on the top-500 Sites
- Started a more complex run with more pages
Staging/Live Spotchecks & Webpages
1) Verify the following webpages that gets pushed to staging and then to the live sites. Look for content messaging, spelling mistakes, broken links & imnages, etc..
- Firstrun page (Staging link)
- Whatsnew Page (Staging link)
- Releasenotes page (Staging link)
2) Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:
- Check for signed installer and applications (windows only)
- Firstrun page, whatsnew page, Home page, Search plugin
- Install, uninstall
- General spotchecks and browser functionality
- Confirm Build IDs
Additional Resource Testing
Given the minimized focus testing around beta 4 due to time constraints, there are a few regression areas that will need some focus testing. This area will call for additional resources to help QA in this effort to ship beta 4 in a timely manner.
- bug 488015, bug 452498 - Test around sites and content that this javascript fix may touch
- bug 475864 - Test around video controls and file input handling
- bug 488497 - Test around Geolocation work
- bug 488015, bug 470487 - Test around crash fixes that need better repro steps
- bug 475066 - Test around tearing away tabs
- bug 486236 - Test around Ts performance
- bug 394759 - Test around Session Restore / undo close window feature
- bug 480169 - Test around new Clear Recent History (sprint) feature
- bug 422526 - Test around localstorage (web developers)
- Fixed P1,P2 1.9.1 bugs from Apri 14th, need to be verified
Fixed Bugs Verifications
The following bugs are fixed for beta 4 and need verifications:
- All Firefox/Platform 1.9.1 bugs: http://tinyurl.com/8guodt
Testday
A beta4 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.
Detailed Results
Smoketests Results
Windows XP [Henrik/Murali/Timr]
- Status: RESULT 4/24/09
- Build: Gecko/20090423 Firefox/3.5b4 ID:20090423204732
- Bugs Found:
- [hskupin] New: bug 490009 - Migration wizard isn't launched after initial installation of Firefox
- Found unclear tests in Forms Manager
- Timr tested signed Win32 builds on XP
- Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4
- Several unclear test cases: bugzilla does not store cookies according to site info dialog - fix test case?
- But b4 is PASS
Mac OSX [Aakash]
- Status: PASS 4/24/09
- Build: Gecko/20090423 Firefox/3.5b4 ID:20090423191946
- Bugs Found:
- Basic Printing test case and Installation are left
- [aakashd] Re-confirmed bug 475066 - dragging a tab out of the browser window doesn't detach the tab
- [timr] Hit bug 461355 - Crash in [HPSmartPrint@0xe1ef] after opening print dialog and canceling
Linux FC7 [Tracy]
- Status - PASS 4/24/09
- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4
- [tracy] Discovered bug 489973 Double confirmation dialog for extension uninstall.
- [aakashd] Still seeing bug bug 485440 - Password Manager dialog window is not resized on Esc Press
- [aakashd] Confirmed bug 426875 - Enable sorting in the 'Page Info' => 'Media' table
BFT Results
Windows XP [henrik]
- Build ID: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4
- Status: PASS
- Issues Noted:
- [hskupin] New: bug 490046 - Geolocation fails when link is dragged into another tab with "update is null"
- [hskupin] bug 469176 - Attempting to drag a tab onto another application creates a new window instead
- [hskupin] bug 366877 - generated microsummaries updated against page loaded into tab even if page is stale
- [hskupin] bug 433748 - XML feed not populated after adding as livemark (Error: this._processor is null)
- [hskupin] bug 127246 Date() gives wrong timezone after time zone change
- [hskupin] New: bug 490198 - Import dialog isn't completely disabled while staying in Private Browsing mode
- [hskupin] New: bug 490199 - Selected option in new privacy pane not sticky when switching from never remember to customized view
Mac OS X [marcia/aakash/tchung]
- Status: PASS
- Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4
- Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4
- Status
- Existing Bugs Seen:
- Issues Noted:
- http://bcheck.scanit.be/bcheck/index.php show an Exception in the Error Console
- Idle Flash player crash running 10.0.r22. Checked B3 and it shows there as well.
- [aakash] filed bug 490088 - Search option for highlighted text is selectable in image context menu
- [aakash] Confirmed bug 175635 - Empty default toolbars do not disappear automatically
- [tchung] Opened bug 490225 - CMD-T opens a new window instead of a new tab when Downloads window is in focus
Linux [FC7,Tracy/Ubuntu,Aakash]
- Status - PASS
- Build Id: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 ID:20090423192432
- Bugs Found:
- [aakashd] Re-confirmed STR for bug 485125 - Crash on close [@ libflashplayer.so@0x4110d ]
Focused Testing
- marcia
- Regression tested Private Browsing, Clear Recent History and Privacy Pane Options, and Geolocation
- Could not reproduce Bug 489885 despite numerous tries.
- Filed Bug 490330
- Regression tested Private Browsing, Clear Recent History and Privacy Pane Options, and Geolocation
Testday Results
- Running Report can be found here until its finalize at the end of the day
- Great Contributors outside MozQA: garrison, DOMi, zpao, FUX, Mavericks, adw, bloodsugarwilksm