QA/Firefox3/TestPlan
Contents
Top Critical issues for QA
This is a working list that QA is using to request be fixed or completed for the Upcoming Fx3 Milestones.
If you have untracked issues in these areas, please enter them here:
- Design still incomplete?
- Major bugs blocking your feature that should be addressed?
- Major bugs blocking overall regression of the trunk that should be fixed before shipping?
Update We've shipped Fx3.0! Next steps is to start tracking 3.1 and 3.0.1 bugs.
Overview
The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.
Major QA Activities:
- Litmus Cleanup
- Litmus Triage Team
- New feature testing - see below
- L10n testing - TBD
- Milestone testing - TBD
Requirements and Ownership
Archives:
Test Areas
Feature Focused Areas
Key: L = Large (3-6 weeks) M = Medium (2-4 weeks) S = Small (1-2 weeks)
Feature Area and Spec | Test Plan Location | QA Contact | Size | Description |
Addons | Addons Test Plan | tchung (new Addons Manager UI, Extensions), stephend (Unified Plugin Manager) | M | Add-on management system, plugin management system, localized user-facing addon text, configuration, updates, Installation, Discovery, FUEL 0.2 |
Content Handling | Microformat Test Plan Web Application Test Plan Download Manager Test Plan |
ctalbert, stephend (Download Manager) | L | microformats, Content Type Processing, download manager, MIME type support, User Interface, Media Plugins (needs definition), firebug (at risk) |
Partner Distribution | TBD | tracy, ctalbert | M | High-Touch/Low-Touch distros, regression, updates, test automation |
OS Platform Integration | Vista Test Plan | Tomcat, marcia | M | Vista Support (At Risk), MSI bundling, Airbag / Breakpad, parental controls (At Risk), Notification, Vista Installer |
Gecko Platform Integration | Gecko Platform Test Plan | martijn, marcia | M | Cairo graphics, nxTextFrame migration to Thebes, Reflow Branch, Caret painting rewrite, Cross-domain XMLHttpRequest (Cocoa widgets covered under Mac Test Plan) |
Mac OS Platform Integration (includes Gecko) | Mac OS Integration Test Plan |
Tomcat, marcia | M | Leopard support, Cocoa implementation, widgets, form controls, Growl Integration on Mac, Mac Installer |
Places | Places Test Plan | Tracy, ABillings | L | Bookmarks, History Indexing, Data Management, User Interface, Extensibility, Performance |
Location Bar | Location Bar Test Plan | Stephend | S | new Location Bar design. Auto-complete |
Search / Tabbed Browsing / Visual Refresh | Search, Tabs, Visual Refresh Test Plan Tab Switching Test Mac Theme Test Plan |
Tomcat (search), Marcia (tabbed browsing) | S | User Interfacing, Animations, Data management |
Security, Privacy / Password, Identity | Security / Privacy Test Plan Password Manager Testplan |
Tomcat, juanb | L | password management, Phishing protection, Private web browsing, Blacklisting (At Risk), password Manager, Password Security |
Breakpad Client/Server | Breakpad Crash Reporting Test Plan | ss | S | Crash reporting tests from client/server, analysis, load |
| ||||
L10N Testing | L10N Test Plan | ctalbert, tchung | S | test automation, new locales, string lengths, bookmarks, rss, search plugins |
L10n Mozilla.com-website Testing | L10n Mozilla.com Test Plan | stephend, pascalc, tchung, timr | M | see test plan |
UI Accessibility | UI Accessibility Test Plan | MarcoZ | S | UI accessibility convention testing |
Major Update Tests (2.0.x -> FF3) | Major Update Test Plan | tchung, juanb, tracy | M | update verification from 2.0.x builds to 3.0b1+. includes upgrades/downgrades/sidegrade cases |
Memory Leaks | Memory Leaks Test Plan | tomcat | M | Checking for memory leaks and asserts on debug builds |
Regression Areas
- Top Web accessibility Tests
- Plugins (eg. Flash, java, AJAX)
- Extensions (eg. Firebug, NoScript, Fotofox)
- Themes (eg. Vista, Joga, Simpsons)
- Top Site Tests (eg. Alexa top 50)
- Financial Sites (eg. banks, online trading)
- Accessibility Tests
- User Performance Testing
- Major / Minor Update Testing
- L10N Testing (may coincide with Feature area)
- Distribution Testing (may coincide with Feature area)
Ownership Needed Litmus Test Case Areas
These are Litmus test case areas that are currently unowned.
- RSS->marcia
- Printing -> Tomcat, tchung
- Location Bar -> Stephend
- Find -> TBD
- Customize Toolbar -> TBD
- Mac Leopard -> marcia
Tiger Team Testing
Daily smoketests are ran against nightly trunk builds. Please see marcia for more information.
Link to Schedule and Assignments
Extensions
For Firefox 2 we had Bug 353206 as extension Tracking bug.
The following extensions were never fixed from their extension compatibly authors to be able to install in Firefox 2.
- Slim Extension List [1]
- Bookmarks Synchronizer 3 [2]
- AutoMarks [3]
- Extended Link Properties [4]
- Snapper [5]
- Cacheout[6]
- Resize Search Box [7]
- superT 0.7.9.1 https://addons.mozilla.org/en-US/firefox/addon/2224
- Reveal extension [8]
Top Crash reports
The main report is the easiest to view.
Tracking Top crashes from Beta 3
- High Priority for beta 4
- Resolved Fixed bug 418377 - crash [@ XPCWrappedNativeScope::FindInJSObjectScope(XPCCallContext&, JSObject*, int)]
- Nom for blocking? please approve bug 418378 - crash [@ nsGlobalWindow::SaveWindowState(nsISupports**)]
- Nom for blocking? please approve bug 418381 - crash [@ HashString(nsAString_internal const&)]
- Lower Priority
- bug 391311 - Topcrash: [@ nsChromeRegistry::CheckForNewChrome] during startup
- bug 418379 - crash [@ nsNavHistoryFolderResultNode::FindChildById(__int64, unsigned int*)]
- bug 418382 - crash [@ UniscribeItem::SaveGlyphs(gfxTextRun*)]
- bug 418384 - crash [@ ClientData::GetOtlTable(long, unsigned char const**, unsigned long*)]
Banking and Financial Sites
For Firefox 3, we’re asking community members and MoCo QA to state which banks and financial institutions they're willing to test. Please add your name to the list.
L10n Testing with Third Parties
For additional string testing, Mozilla contracts a third party company to utilize language experts with knowledge of Testing. Results are listed in the links here.
Review Feedback
In this section, we will roll up all common feedback pulled from external sources that appears common and should be tracked by QA. Feedback sources can include:
- Hendrix Feedback system (mozilla.feedback newsgroups)
- Bugzilla
- Breakpad Crash Reporter
- Mozillazine Forums
Feedback summary and review notes are posted here.
Flash Beta plugin Testing
- http://weblogs.macromedia.com/labs/archives/2007/11/flash_player_9_6.cfm
- Team members actively testing betas
- marcia
- abillings
- stephend
Post Mortem Notes
The following is a collection of post mortem notes that the team has collected over the course of the release.
Additional References
New FF3 Feature list - An incomplete list of what Firefox 3 developers are working on.