« Firefox 10 | Firefox 11 | Firefox 12 »

Firefox 11 Test Plan

Summary

Milestone	Firefox 11
Status	Currently in Beta -- Release on March 13, 2012
Lead	Anthony Hughes (irc: ashughes)
2nd	Juan Becerra (irc: juanb)
Builds	HTTPS

The following is the test plan for Firefox 11 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 11. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

2011-11-10: [DONE] Nightly Merge
2011-12-22: [DONE] Aurora Merge
2012-02-03: [DONE] Beta 1
2012-02-07: [ON TRACK] Beta 2
2012-02-14: {{|}} Beta 3
2012-02:21: {{|}} Beta 4
2012-02-28: {{|}} Beta 5
2012-03-06: {{|}} Beta 6
2012-03-09: {{|}} Release Candidate
2012-03-13: {{|}} Final

Features

See the Feature Release Tracking wiki for a general overview

[DONE]

Chrome Migration (Paul Silaghi): SIGNED OFF on Firefox 11 RC build 1 on 2012-03-12
SPDY (Ioana Budnar): SIGNED OFF on Firefox 11 RC build 2 on 2012-03-13
Page Inspector Tilt (Alin Trif): SIGNED OFF 2012-01-25
Add-on Sync (Tracy Walker): SIGNED OFF 2012-01-25
- NOTE: will not sync across application versions, add-ons outside of AMO, plugins, personas, and search plugins

[ON TRACK]

Style Editor (Alex Lakatos): 2012-01-31
- Pending uplift of some bug fixes bug 720468 (Outstanding bugs)

[AT RISK]

[MISSED]

Home Tab (Vlad Ghetiu): delayed to Firefox 12
Media Controls Refresh (Mihaela Velimiroviciu): delayed to Firefox 12
Improve Awesomebar Results (Simona Bidau): delayed to Firefox 12
UAC Service (Simona Bidau): delayed to Firefox 12
New Tab Page (Virgil Dicu): delayed to Firefox 12
TLS Telemetry (unowned): delayed to Firefox 13
In-content UI Unification (Ioana Budnar): unscoped, no milestone

Final Sign-off

Litmus testrun is being built (to be run in week leading up to Beta merge

Endurance Automation

Endurance automation is run every day during Nightly. The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

Owner: Mihaela Velimiroviciu
Status: SIGNED OFF
Reports: dashboard, spreadsheet
Issues: none found

Functional Automation

Functional automation is run every day on Nightly and Aurora. The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

Owner: Vlad Maniac (2nd - Alex Lakatos)
Status: on track
Reports: dashboard
Issues: none found

Bug Triage

This section tracks bug triage work. The purpose is to triage UNCONFIRMED incoming bugs during Nightly and Aurora. Once fixed, they will be triaged and verified during Beta.

Owner: Vlad Ghetiu
Status: on track
Queries: UNCONFIRMED, REOPENED, NEW
Issues: none found

Aurora Merge

Functional Automation

Platform	Result: report
Windows 2000	PASS - 2011-12-22
Windows XP	PASS - 2011-12-22
Windows Vista	PASS - 2011-12-22
Windows 7 32-bit	PASS - 2011-12-22
Windows 7 64-bit	PASS - 2011-12-22
Mac OS X 10.6	PASS - 2011-12-22
Mac OS X 10.7	PASS - 2011-12-22
Ubuntu 11.04 32-bit	PASS - 2011-12-22
Ubuntu 11.04 64-bit	PASS - 2011-12-22

Updates

Platform	Result
Auroratest
Win32	Mozilla/5.0 (Windows NT 5.1; rv:10.0a2) Gecko/20111211 Firefox/10.0a2 Mozilla/5.0 (Windows NT 5.1; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Mac	Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0a2) Gecko/20111215 Firefox/10.0a2 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Linux	Mozilla/5.0 (X11; Linux i686 on x86_64; rv:10.0a2) Gecko/20111125 Firefox/10.0a2 Mozilla/5.0 (X11; Linux i686 on x86_64; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Aurora
Win32	Mozilla/5.0 (Windows NT 5.1; rv:10.0a2) Gecko/20111211 Firefox/10.0a2 Mozilla/5.0 (Windows NT 5.1; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Mac	Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0a2) Gecko/20111215 Firefox/10.0a2 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Linux	Mozilla/5.0 (X11; Linux i686 on x86_64; rv:10.0a2) Gecko/20111125 Firefox/10.0a2 Mozilla/5.0 (X11; Linux i686 on x86_64; rv:11.0a2) Gecko/20111222 Firefox/11.0a2

Beta 1

Builds: HTTPS
Litmus: testrun, report

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications (10%)
[DONE] Check with Support for top issues from previous Beta/Release
[DONE] Check with Marcia for top-crashers spiking
[DONE] Update tests on releasetest
[DONE] Get final sign-off from Bob Moss
[DONE] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Assignee	Result: report
Windows 2000	ashughes	PASS - 2012-02-01
Windows XP	ashughes	PASS - 2012-02-01
Windows Vista	ashughes	PASS - 2012-02-01
Windows 7 32-bit	ashughes	PASS - 2012-02-01
Windows 7 64-bit	ashughes	PASS - 2012-02-01
Mac OS X 10.6	ashughes	PASS - 2012-02-01
Mac OS X 10.7	ashughes	PASS - 2012-02-01
Ubuntu 11.10 32-bit	ashughes	PASS - 2012-02-01
Ubuntu 11.10 64-bit	ashughes	PASS - 2012-02-01

Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform	Assignee	Result
Windows 2000	Simona	PASS - 2012-02-02
Windows XP	Ioana	PASS - 2012-02-02
Windows Vista	Simona	PASS - 2012-02-02
Windows 7 32-bit	Mihaela	PASS - 2012-02-02
Windows 7 64-bit	Paul	PASS - 2012-02-02
Mac OS X 10.6.8	VladG	PASS - 2012-02-02
Mac OS X 10.7.2	VladG	PASS - 2012-02-02
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-02-02
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-02-02

Results in Litmus

Focus Testing

The following are the detailed results of the focused feature testing. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

[DONE] Rich Profile Updates Etherpad
[DONE] Add-on Compatibility (including default compatible) Etherpad
[DONE] Performance (typing and scrolling on intense pages -- like Facebook timeline) Etherpad
[DONE] Web Compatibility (bank sites, social networking, video and audio streaming, etc) Etherpad
[DONE] Anti-virus Compatibility (checking that AV software does not interfere, degrade Firefox performance) Etherpad
[DONE] Plugin Stability (games & videos, flash, java, silverlight, etc) Etherpad
[DONE] Run the Software Update and Silent Update FFTs

Feature Sign-off Testing

The following are the results of feature testing for those features which are shipping. Use the Firefox 11.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.

Feature	Owner	Win32	Mac	Linux
Chrome Migration	Paul Silaghi	PASS - 2012-02-02	PASS - 2012-02-02	PASS - 2012-02-02
SPDY	Ioana Budnar	PASS - 2012-02-02	PASS - 2012-02-02	PASS - 2012-02-02
Page Inspector Tilt	Alin Trif	PASS - 2012-02-02	PASS - 2012-02-02	PASS - 2012-02-02
Add-on Sync	Tracy Walker	PASS - 2012-02-02	PASS - 2012-02-02	PASS - 2012-02-02

Update Tests

The following are details of update verifications

Platform	Releasetest: report	Beta: report
Windows 2000	PASS - 2012-02-02	PASS - 2012-02-03
Windows XP	PASS - 2012-02-02	PASS - 2012-02-03
Windows Vista	PASS - 2012-02-02	PASS - 2012-02-03
Windows 7 32-bit	PASS - 2012-02-02	PASS - 2012-02-03
Windos 7 64-bit	PASS - 2012-02-02	PASS - 2012-02-03
Mac OS X 10.6	PASS - 2012-02-02	PASS - 2012-02-03
Mac OS 10.7	PASS - 2012-02-02	PASS - 2012-02-03
Ubuntu 11.10 32-bit	PASS - 2012-02-02	PASS - 2012-02-03
Ubuntu 11.10 64-bit	PASS - 2012-02-02	PASS - 2012-02-03

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]

Queries

QA+ - used by testers to triage and verify fixes
- UNTRIAGED: 102, VERIFIED: 12
- Softvision verifications etherpad
QA+ SG - used by the Security team to triage and verify security fixes
- UNTRIAGED: 30, VERIFIED: 0
status-firefox11:fixed - used by QA lead to triage for potential QA+ bugs
- QA?: 13, QA+: 52, QA-: 26

Issues Found

Bug	Status
bug 723444 - Angry Birds is running jerky	Need help investigating, should not block 11b1
bug 723615 - Fix Mozilla UA sniffing	results "update to Firefox 10" what's new page
bug 723920 - Firefox 10beta does not auto-update	INVALID: updates download in the background, prompt on completion

Known Issues

Bug	Status
bug 681922 In google docs Upload -> Files has no highlighting	Not a Firefox bug, regressed due to Google back-out
bug 697696 Highlight issue in amazon's password box	Needs reduced testcase for debugging, no functional deficiency
bug 692446 Firefox prompts to install WMP plug-in though it's installed	Still investigating, might be issue with Microsoft Installer
bug 702045 Make Firefox able to survive a Windows System Restore	Development work still ongoing
bug 278860 confusing "profile in use"/"already running" error when profile is missing (not found)	NEW

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 2

Builds: HTTPS
Litmus: testrun, report

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications (20%)
[DONE] Check with Support for top issues from previous Beta/Release
[DONE] Check with Marcia for top-crashers spiking
[DONE] Update tests on releasetest
[DONE] Get final sign-off from Bob Moss
[DONE] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Assignee	Result: report
Windows 2000	ashughes	PASS - 2012-02-08
Windows XP	ashughes	PASS - 2012-02-08
Windows Vista	ashughes	PASS - 2012-02-08
Windows 7 32-bit	ashughes	PASS - 2012-02-08
Windows 7 64-bit	ashughes	PASS - 2012-02-08
Mac OS X 10.6	ashughes	PASS - 2012-02-08
Mac OS X 10.7	ashughes	PASS - 2012-02-08
Ubuntu 11.10 32-bit	ashughes	PASS - 2012-02-08
Ubuntu 11.10 64-bit	ashughes	PASS - 2012-02-08

Test Failures

testSecurity/testSubmitUnencryptedInfoWarning: report (bug 725486)

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform	Assignee	Result
Windows 2000	Simona	PASS - 2012-02-10
Windows XP	Remus	PASS - 2012-02-09
Windows Vista	Simona	PASS - 2012-02-10
Windows 7 32-bit	AlinT	PASS - 2012-02-09
Windows 7 64-bit	Paul	PASS - 2012-02-10
Mac OS X 10.6.8	Alex	PASS - 2012-02-09
Mac OS X 10.7.2	Alex	PASS - 2012-02-09
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-02-10
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-02-10

Results in Litmus

Focus Testing

Run the Fx Firefox 11.0 BFT - Regression Tests from the Litmus testrun and then do exploratory testing around those areas. Indicate any detailed results or issues found below.

Platform	Assignee	Result
Windows XP	Virgil	PASS - 2012-02-10
Windows Vista	Simona	PASS - 2012-02-10
Windows 7 32-bit	Ioana	PASS - 2012-02-10
Windows 7 64-bit	Paul	PASS - 2012-02-10
Mac OS X 10.6.8	Mihaela	PASS - 2012-02-10
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-02-10
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-02-10

Update Tests

The following are details of update verifications

Platform	Releasetest: report	Beta: report
Windows 2000	PASS - 2012-02-09	PASS - 2012-02-10
Windows XP	PASS - 2012-02-09	PASS - 2012-02-10
Windows Vista	PASS - 2012-02-09	PASS - 2012-02-10
Windows 7 32-bit	PASS - 2012-02-09	PASS - 2012-02-10
Windos 7 64-bit	PASS - 2012-02-09	PASS - 2012-02-10
Mac OS X 10.6	PASS - 2012-02-09	PASS - 2012-02-10
Mac OS 10.7	PASS - 2012-02-09	PASS - 2012-02-10
Ubuntu 11.10 32-bit	PASS - 2012-02-09	PASS - 2012-02-10
Ubuntu 11.10 64-bit	PASS - 2012-02-09	PASS - 2012-02-10

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Softvision verifications spreadsheet

Queries

Bugs	Status
QA+: non-security bugs which need verification	VERIFIED: 7 / 90
QA+ SG: security bugs which need verification	VERIFIED: 0 / 31
status-firefox11:fixed: bugs needing triage for QA+	TRIAGED: 24 / 24
P1 Verifications
bug 718939 Java applet causes text entry fields to become semi-unresponsive	VERIFIED FIXED (juanb)
bug 724284 Use after free in nsXBLDocumentInfo::ReadPrototypeBindings	PENDING (marcia)

Issues Found

Bug	Status
bug 726028 Crash in firefox with nssCertificate_Destroy-not reproducible	Needs investigation
bug 725975 Adobe flash plugin crashes when swithching full screen mode on/off (CMD+SHIFT+F) with focus on flash content (video) (Single Core Processor)	Investigating, reproducible back Firefox 6
bug 725296 Browser is unresponsive while Silverlight content is loading	Investigating, reproducible back to 3.6
bug 725631 Forward button disabled after navigating back to Youtube	Not widely reproducible, geo-locale specific?
bug 725614 Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content	Not a regression, reproducible back to 3.6.26
bug 723444 Angry Birds is running jerky	Firefox 11 regression, mozilla-central 2011-11-21
bug 702045 Make Firefox able to survive a Windows System Restore	Development work ongoing
bug 697696 Highlight issue in amazon's password box	Tech Evangelism: Amazon's bug
bug 681922 In google docs Upload -> Files has no highlighting	Tech Evangelism: Google's bug
bug 692446 Firefox prompts to install windows media player plugin even though it is already installed	Issue with MSI, not strictly a Firefox bug
bug 638770 clicking on a quicktime video pauses it	Trivial UX issue
bug 709029 The RealPlayer plugin crashes when trying to play a RealPlayer video	Very old regression going back to 4.0 (OOPP?)
bug 590963 Google Maps - Earth View Causes Freeze	Very old regression going back to 4.0b4

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 3 build 2

Builds: HTTPS
Litmus: testrun, report
Mozmill: [ report]

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications
[DONE] Update tests on releasetest
{{ |}} Check with Marcia for top-crashers spiking
{{ |}} Get final sign-off from Bob Moss
{{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows 2000	PASS - 2012-02-16
Windows XP	PASS - 2012-02-16
Windows Vista	PASS - 2012-02-16
Windows 7 32-bit	PASS - 2012-02-16
Windows 7 64-bit	PASS - 2012-02-16
Mac OS X 10.6	PASS - 2012-02-16
Mac OS X 10.7	PASS - 2012-02-16
Ubuntu 11.10 32-bit	PASS - 2012-02-16
Ubuntu 11.10 64-bit	PASS - 2012-02-16

Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoktests tests.

Platform	Assignee	Result: report
Windows 2000	Virgil	PASS - 2012-02-16
Windows XP	Ioana	PASS - 2012-02-16
Windows Vista	Simona	PASS - 2012-02-16
Windows 7 32-bit	Mihaela	PASS - 2012-02-16
Windows 7 64-bit	ashughes	PASS - 2012-02-16
Mac OS X 10.6.8	Vlad	PASS - 2012-02-16
Mac OS X 10.7.3	Vlad	PASS - 2012-02-16
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-02-16
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-02-16

Focus Tests

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

[DONE] PNG handling exploratory testing Etherpad

Platform	Assignee	Result: report
Windows 2000	tester	result - YYYY-MM-DD
Windows XP	tester	result - YYYY-MM-DD
Windows Vista	tester	result - YYYY-MM-DD
Windows 7 32-bit	tester	result - YYYY-MM-DD
Windows 7 64-bit	tester	result - YYYY-MM-DD
Mac OS X 10.6.8	tester	result - YYYY-MM-DD
Mac OS X 10.7.3	tester	result - YYYY-MM-DD
Ubuntu 11.10 32-bit	tester	result - YYYY-MM-DD
Ubuntu 11.10 64-bit	tester	result - YYYY-MM-DD

NOTE: Prioritize Windows 7, XP, Mac OS X 10.6, and Ubuntu 32-bit -- do other platforms only as time permits

Update Tests

The following are details of update verifications

Platform	Releasetest: report	Release: report
Windows 2000	PASS - 2012-02-16	PASS - 2012-02-17
Windows XP	PASS - 2012-02-16	PASS - 2012-02-17
Windows Vista	PASS - 2012-02-16	PASS - 2012-02-17
Windows 7 32-bit	PASS - 2012-02-16	PASS - 2012-02-17
Windos 7 64-bit	PASS - 2012-02-16	PASS - 2012-02-17
Mac OS X 10.6	PASS - 2012-02-16	PASS - 2012-02-17
Mac OS 10.7	PASS - 2012-02-16	PASS - 2012-02-17
Ubuntu 11.10 32-bit	PASS - 2012-02-16	PASS - 2012-02-17
Ubuntu 11.10 64-bit	PASS - 2012-02-16	PASS - 2012-02-17

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Softvision verifications spreadsheet

Queries

Bugs	Status
P1 bug 727401 libpng: integer overflow leading to heap-buffer overflow	VERIFIED FIXED
QA+: non-security bugs which need verification	VERIFIED: 9 / 90
QA+ SG: security bugs which need verification	VERIFIED: 2 / 31
status-firefox11:fixed: bugs needing triage for QA+	TRIAGED: 0

Issues Found

Bug	Status

Previously Known
bug 725982 Flash player stops rendering on right click	NEW.Needs investigation
bug 726028 Crash in firefox with nssCertificate_Destroy-not reproducible	Needs investigation
bug 725975 Adobe flash plugin crashes when swithching full screen mode on/off (CMD+SHIFT+F) with focus on flash content (video) (Single Core Processor)	Investigating, reproducible back Firefox 6
bug 725296 Browser is unresponsive while Silverlight content is loading	Investigating, reproducible back to 3.6
bug 725631 Forward button disabled after navigating back to Youtube	Not widely reproducible, geo-locale specific?
bug 725614 Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content	Not a regression, reproducible back to 3.6.26
bug 723444 Angry Birds is running jerky	Firefox 11 regression, mozilla-central 2011-11-21
bug 702045 Make Firefox able to survive a Windows System Restore	Development work ongoing
bug 697696 Highlight issue in amazon's password box	Tech Evangelism: Amazon's bug
bug 681922 In google docs Upload -> Files has no highlighting	Tech Evangelism: Google's bug
bug 692446 Firefox prompts to install windows media player plugin even though it is already installed	Issue with MSI, not strictly a Firefox bug
bug 638770 clicking on a quicktime video pauses it	Trivial UX issue
bug 709029 The RealPlayer plugin crashes when trying to play a RealPlayer video	Very old regression going back to 4.0 (OOPP?)
bug 590963 Google Maps - Earth View Causes Freeze	Very old regression going back to 4.0b4
bug 717521 Moving the map in Bing Maps locks my X11	NEW

Beta 4

Builds: HTTPS
Litmus: testrun, report
Mozmill: dashboard

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications
[DONE] Update tests on releasetest
[DONE] Check with Marcia for top-crashers spiking
[MISSED] Get final sign-off from Bob Moss
{{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows 2000	PASS - 2012-02-22
Windows XP	PASS - 2012-02-22
Windows Vista	PASS - 2012-02-22
Windows 7 32-bit	PASS - 2012-02-22
Windows 7 64-bit	PASS - 2012-02-22
Mac OS X 10.6	PASS - 2012-02-22
Mac OS X 10.7	PASS - 2012-02-22
Ubuntu 11.10 32-bit	PASS - 2012-02-22
Ubuntu 11.10 64-bit	PASS - 2012-02-22

Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoktests tests.

Platform	Assignee	Result: report
Windows 2000	Simona	PASS - 2012-02-23
Windows XP	Ioana	PASS - 2012-02-23
Windows Vista	Simona	PASS - 2012-02-23
Windows 7 32-bit	Mihaela	PASS - 2012-02-23
Windows 7 64-bit	Paul	PASS - 2012-02-23
Mac OS X 10.6.8	Vlad	PASS - 2012-02-23
Mac OS X 10.7.3	Vlad	PASS - 2012-02-23
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-02-23
Ubuntu 11.10 64-bit	Ioana	PASS - 2012-02-23

Focus Tests

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform	Assignee	Result: report
Windows XP	Paul	PASS - 2012-02-23
Windows 7 32-bit	Mihaela	PASS - 2012-02-23
Mac OS X 10.6.8	Vlad	PASS - 2012-02-23
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-02-23

NOTE: Prioritize Windows 7, XP, Mac OS X 10.6, and Ubuntu 32-bit -- do other platforms only as time permits

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b5 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b4. This is a temporary stop-gap to ease burden on RelEng resources.

Platform	Releasetest: report	Beta: report
Windows 2000	PASS - 2012-02-23	PASS - 2012-02-24
Windows XP	PASS - 2012-02-23	PASS - 2012-02-24
Windows Vista	PASS - 2012-02-23	PASS - 2012-02-24
Windows 7 32-bit	PASS - 2012-02-23	PASS - 2012-02-24
Windows 7 64-bit	PASS - 2012-02-23	PASS - 2012-02-24
Mac OS X 10.6	PASS - 2012-02-23	PASS - 2012-02-24
Mac OS 10.7	PASS - 2012-02-23	PASS - 2012-02-24
Ubuntu 11.10 32-bit	PASS - 2012-02-23	PASS - 2012-02-24
Ubuntu 11.10 64-bit	PASS - 2012-02-23	PASS - 2012-02-24

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Softvision verifications spreadsheet

Queries

Bugs	Status
QA-WANTED: bugs which need QA to reproduce	TRIAGED: 12 / 12
QA+: non-security bugs needing verification	VERIFIED: 22 / 72
QA+ SG: security bugs which need verification	VERIFIED: 1 / 36
status-firefox11:fixed: bugs needing triage for QA+	TRIAGED: 25

Issues

New

If you find any bugs which are not listed in the Tracking section below, please file them in bugzilla and link them below.

bug 730285 - No update channel is found after restoring the OS to a previous point
- INVESTIGATING: Seems to only affect those who change their channel
bug 729872 - HTML5 fullscreen loosing focus causes text entry fields to become unresponsive
- INVESTIGATING: Regression in Nightly caused by bug 685402 (2011-11-04)
bug 729907 - IMDB videos lose their image and controls after restart
- TRACKING-: Affects back to Firefox 3.6, reloading the page works around the issue
bug 547152 - Stray RealPlayer plugin frame appears after switching to another tab
- TRACKING-: Affects back to Firefox 2.0

Tracking

investigating

[RESOLVED] bug 725982 - Shockwave player stops rendering on right click
- Fixed by updating Shockwave plugin to the latest
bug 726028 - Crash in nssCertificate_Destroy
- Still investigating, so far unreproducible (2012-02-22)
bug 725975 - Flash crashes switching full screen mode
- Still investigating, so far unreproducible (2012-02-22)
bug 725631 - Forward button disabled after navigating back to Youtube
- WONTFIX? (2012-02-22)

tracking?

bug 725296 - Hang while Silverlight content is loading
- Also affects 3.6, Chrome (2012-02-22)
bug 725614 - Shortcut keys fail for pages containing Java/Shockwave/PDF after focus on browser content
- Also affects 3.6 (2012-02-22)
bug 692446 - Firefox prompts to install windows media player plugin even though it is already installed
- Not a Firefox bug, devs working on workaround (2012-02-22)
bug 709029 - RealPlayer plugin crashes
- Started with OOPP (2012-02-22)
bug 590963 - Google Maps Earth View causes hang
- Firefox 4 regression (2012-02-22)
bug 717521 - Moving the map in Bing Maps locks X11
- Regression from bug 702739

tracking+

bug 723444 - Angry Birds is running jerky [ASSIGNED]
- Working with devs by testing TRY builds
bug 702045 - Make Firefox able to survive a Windows System Restore [ASSIGNED]
- Last status update was Nov'11 (2012-02-22)
bug 715401 - Firefox Crash @ nsCanvasRenderingContext2DAzure::ClearRect with Yahoo Toolbar
- QA-WANTED

Beta 5

Builds: HTTPS
Litmus: testrun, report
Mozmill: dashboard

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications
[DONE] Update tests on releasetest
[MISSED] Check with Marcia for top-crashers spiking (PTO)
[DONE] Get final sign-off from Bob Moss
[DONE] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows 2000	PASS - 2012-02-29
Windows XP	PASS - 2012-02-29
Windows Vista	PASS - 2012-02-29
Windows 7 32-bit	PASS - 2012-02-29
Windows 7 64-bit	PASS - 2012-02-29
Mac OS X 10.6	PASS - 2012-02-29
Mac OS X 10.7	PASS - 2012-02-29
Ubuntu 11.10 32-bit	PASS - 2012-02-29
Ubuntu 11.10 64-bit	PASS - 2012-02-29

Test Failures

bug 731690 - testPreferences/testPreferredLanguage.js: possible Google regression

Manual Tests

Smoketests

The following are the detailed results of the manual Smoktests tests.

Platform	Assignee	Result: report
Windows 2000	Simona	PASS - 2012-03-01
Windows XP	Ioana	PASS - 2012-03-01
Windows Vista	Simona	PASS - 2012-03-01
Windows 7 32-bit	Mihaela	PASS - 2012-03-01
Windows 7 64-bit	Paul	PASS - 2012-03-01
Mac OS X 10.6.8	VladG	PASS - 2012-03-01
Mac OS X 10.7.3	VladG	PASS - 2012-03-01
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-03-01
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-03-01

Focus Tests

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform	Assignee	Result: report
Windows 7 32-bit	Mihaela	PASS - 2012-03-01
Mac OS X 10.6.8	VladG	PASS - 2012-03-01
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-03-01
Windows XP	Ioana, Mihaela	PASS - 2012-03-02
MacOS X 10.7.3	VladG	PASS - 2012-03-02
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-03-02
Windows 7 64-bit	Paul	PASS - 2012-03-02

NOTE: Prioritize Windows 7, Mac OS X 10.6, and Ubuntu 32-bit -- do other platforms only as time permits

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b5 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b4. This is a temporary stop-gap to ease burden on RelEng resources.

Platform	Releasetest: report	Beta: report
Windows 2000	PASS - 2012-03-01	PASS - 2012-03-02
Windows XP	PASS - 2012-03-01	PASS - 2012-03-02
Windows Vista	PASS - 2012-03-01	PASS - 2012-03-02
Windows 7 32-bit	PASS - 2012-03-01	PASS - 2012-03-02
Windows 7 64-bit	PASS - 2012-03-01	PASS - 2012-03-02
Mac OS X 10.6	PASS - 2012-03-01	PASS - 2012-03-02
Mac OS 10.7	PASS - 2012-03-01	PASS - 2012-03-02
Ubuntu 11.10 32-bit	PASS - 2012-03-01	PASS - 2012-03-02
Ubuntu 11.10 64-bit	PASS - 2012-03-01	PASS - 2012-03-02

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Softvision verifications spreadsheet

Queries

QA+: non-security bugs needing verification (10 / 11 VERIFIED)
QA+ SG: security bugs which need verification (0 / 33 VERIFIED)
status-firefox11:fixed: bugs needing triage for QA+ (0 / ? TRIAGED)
UNCO: unconfirmed bugs filed during Firefox 11 (0 / 38 TRIAGED)

Issues

New

Bug	Status
bug 731981 - Missing steps for missing Shockwave plugin notification	Tech Evangelism - website bug due to user-agent
bug 732388 - Firefox crash ffi_call (Roboform Toolbar 7.7.3.0)	Need investigating

QA Investigating (QA-WANTED)

Bug	Status
bug 720679 - Crash @ mozilla::dom::workers::WorkerPrivate::CancelAllTimeouts	Marcia was investigating, does bsmedbergs feedback help?
bug 701404 - Silverlight crashiness using Bing Video	Vlad needs to follow-up with testing Silverlight 5 64-bit
bug 716391 - Crash @je_free xul.dll@0xbb...f	Marcia was investigating, status update?
bug 723444 - Angry Birds is running jerky	Need to do comparison with Firefox 10 (Paul:SV)

Engineering Investigating

Bug	Status
bug 729872 - HTML5 fullscreen breaks text entry carat	ASSIGNED, Regression from bug 685402
bug 692446 - WMP plugin repeatedly asks to install	ASSIGNED
bug 717521 - Moving the map in Bing Maps locks X11	ASSIGNED, Regression from bug 702739
bug 730285 - Can't update Firefox after Windows system restore	ASSIGNED
bug 715401 - Crash @nsCanvasRenderingContext2DAzure::ClearRect (Yahoo Toolbar)	Bas Schouten is investigating
bug 671468 - Crash @nsSocketOutputStream::Write (Yandex)	Over to Honza for STR
bug 711656 - Crash in TextStageManager::AddStagesForSubrect w/Intel GMA4500HD	Benoit Jacob is investigating
bug 731720 - Hang logging into Evernote	Alex Keybl is investigating
bug 656758 - Crash @nsSocketInputStream::Read (nsUrlClassifierDBService)	Josh Aas is investigating

Followup Needed

Day 2 Testing

Ideas for Testing:

looking through unconfirmed bugs filed against Firefox 11 for possible regressions (http://bit.ly/wps2eJ)
- Triaged bugs spreadsheet
regression tests on other platforms (WinXP, OSX 10.7, Ubuntu 64-bit, Windows 7 64-bit in that order)
exploratory testing against top-sites (Facebook, Youtube, webmail, banking, etc)
exploratory testing with top addons
default compatible add-ons (installing add-ons in a previous beta which are not compatible with 11 and checking that they still work after updating to Firefox 11 on releasetest)
exploratory testing for performance issues (GC/CC times, hangs, crashes, etc)

Blocklist Testing

When testing the following blocklist bugs, please record results in this spreadsheet and provide any specific details below.

bug 715744 - DLL block request BExternal.dll

Software
- Babylon 9 Free
Steps to Perform:
- Install Firefox 11.0b5
- Install the Babylon Software
- Start Firefox and verify the Babylon Toolbar is installed
- Repeat with the try build
- Verify the Babylon software has been blocklisted
Results

bug 665775 - Blocklist Extensions and dlls to prevent datamngr.dll crashes

Software
- JZip (should include JZip Toolbar)
- Bandoo (should include MediaBar/SearchQu Toolbar)
- iMesh (should include iMesh Toolbar)
- BearShare (should include BearShare Toolbar)
- Try to track down older versions of the software as well
- Try with different mixes (one, many, or all of the listed software)
Steps to Perform:
- Install Firefox 11.0b5
- Install some of the above software
- Start Firefox and verify the add-ons are installed
- Repeat with the [ try] build
- Verify the add-ons are blocklisted
Results

Default to Compatible

See bug 729654

Automation

4.0b7-b12, 4.0, 4.0.1, 5.0b*, 5.0, 5.0.1, 6.0b*, 6.0, 7.0b*, 8.0b*, 9.0b* -> 11.0b5

Platform	Releasetest: report	Beta: report
Windows 2000	PASS - 2012-03-05	PASS - 2012-03-06
Windows XP	PASS - 2012-03-05	PASS - 2012-03-06
Windows Vista	PASS - 2012-03-05	PASS - 2012-03-06
Windows 7 32-bit	PASS - 2012-03-05	PASS - 2012-03-06
Windows 7 64-bit	PASS - 2012-03-05	PASS - 2012-03-06
Mac OS X 10.6	PASS - 2012-03-05	PASS - 2012-03-06
Mac OS 10.7	PASS - 2012-03-05	PASS - 2012-03-06
Ubuntu 11.10 32-bit	PASS - 2012-03-05	PASS - 2012-03-06
Ubuntu 11.10 64-bit	PASS - 2012-03-05	PASS - 2012-03-06

Manual (Releasetest)

one add-on with maxVer for each release background updates to 11.0b5 (you can re-use add-ons from this spreadsheet)
Results (Google Spreadsheet)

Addons	Windows	Mac	Linux
Firefox 9.* Cooliris, AniWeather, IE Tab Mix Plus, Evernote Web Clipper, Fire Shot, Download Flash and Video, Interface pt-BR, Awesome Screenshot, Foxy Tunes, Video Surf	PASS 2012-03-05	PASS 2012-03-05	PASS 2012-03-05
Firefox 8.* English South African Language Pack, Romanian Language Pack, Afrikaans Language Pack, Esperanto Language Pack, Edit Cookies, Leo Search, Zulu Language Pack, Northern Soho Language Pack, Start Google Plus, Auto Shutdown NG	PASS 2012-03-05	PASS 2012-03-05	PASS 2012-03-05
Firefox 7.* Banco do Brasil, Adicional de Seguranca, Fire DIff, Fire Tray, Fire Bomb, Sinafox, Zoom Fox, UrlBarExt, Metal + Glass, Fantapper	PASS 2012-03-05	PASS 2012-03-05	PASS 2012-03-05
Firefox 6.* Screen Grab, Capture Fox, Maximum Adblock, Smart Video for Youtube, aFewClicksAway, Mozilla Labs Lab Kit, Image Fit Plus7Fox, Validate Me, MatchBox	PASS 2012-03-05	PASS 2012-03-05	PASS 2012-03-05
Firefox 5.* kwi Click, Multirow Bookmarks Toolbar, Glue Book Recommendations, Google Sharing, Google Bar, ErrorZilla Plus, Facebook Blocker, Murdoch Alert, Cookie Safe, ESPN Cricinfo	PASS 2012-03-05	PASS 2012-03-05	PASS 2012-03-05
Firefox 4.* Fast Dial, Price Gong, Irvine Context Menu, Greased Lightbox, SEO Toolbar, SEOpen, Clip to DEVONthink, ViewInFirefox, Native XP, Switch Hosts	PASS 2012-03-05	PASS 2012-03-05	PASS 2012-03-05

Manual (Beta)

Manual Updates
- Install a previous version of Firefox
- Install an add-on which is max-compatible with that version (ie. not compatible with Firefox 11.0)
- Set update channel to beta
- Manually check for and install an update
Repeat the above for Automatic Update after setting these preferences:
- Set app.update.log = true
- Set app.update.interval = 10
- Set app.update.idletime = 0
- Set app.update.download.backgroundInterval = 10
- Restart and open Error Console
- Once the update is completed downloading in the background, restart Firefox
EXPECTED:
- You should not be prompted to enable/install any add-ons
- You should successfully update to Firefox 11.0b5
- Your addons should be installed and enabled in the Add-ons Manager
RESULTS:

Note - we will also run our full-suite DTC automation once we have RC updates (~ March 12)

Beta 6

Builds: HTTPS
Litmus: testrun, report

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications
[DONE] Updates on releasetest
[DONE] Peer sign-off
[DONE] Updates on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows 2000	PASS - 2012-03-06
Windows XP	PASS - 2012-03-06
Windows Vista	PASS - 2012-03-06
Windows 7 32-bit	PASS - 2012-03-06
Windows 7 64-bit	PASS - 2012-03-06
Mac OS X 10.6	PASS - 2012-03-06
Mac OS X 10.7	PASS - 2012-03-06
Ubuntu 11.10 32-bit	PASS - 2012-03-06
Ubuntu 11.10 64-bit	PASS - 2012-03-06

Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoketests tests.

Platform	Assignee	Result: report
Windows 2000	Simona	PASS - 2012-03-06
Windows XP	Paul	PASS - 2012-03-06
Windows Vista	Simona	PASS - 2012-03-06
Windows 7 32-bit	Mihaela	PASS - 2012-03-06
Windows 7 64-bit	Paul	PASS - 2012-03-06
Mac OS X 10.6.8	VladG	PASS - 2012-03-06
Mac OS X 10.7.3	VladG	PASS - 2012-03-06
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-03-07
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-03-06

Focus Tests

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform	Assignee	Result: report
Windows 7 32-bit	Mihaela	PASS - 2012-03-06
Mac OS X 10.6.8	VladG	PASS - 2012-03-07
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-03-06
Windows XP	Paul	PASS - 2012-03-06

NOTE: Prioritize Windows 7, Mac OS X 10.6, and Ubuntu 32-bit -- do other platforms only as time permits

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Softvision verifications spreadsheet

Queries

QA+: non-security bugs needing verification (9 / 10 VERIFIED)
QA+ SG: security bugs which need verification (15 / 36 VERIFIED)
status-firefox11:fixed: bugs needing triage for QA+ (49 / 49 TRIAGED)
UNCO: unconfirmed bugs filed during Firefox 11 (0 / 38 TRIAGED)

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b5 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b4. This is a temporary stop-gap to ease burden on RelEng resources.

Platform	Releasetest: report	Beta: report
Windows 2000	PASS - 2012-03-06	PASS - 2012-03-07
Windows XP	PASS - 2012-03-06	PASS - 2012-03-07
Windows Vista	PASS - 2012-03-06	PASS - 2012-03-07
Windows 7 32-bit	PASS - 2012-03-06	PASS - 2012-03-07
Windows 7 64-bit	PASS - 2012-03-06	PASS - 2012-03-07
Mac OS X 10.6	PASS - 2012-03-06	PASS - 2012-03-07
Mac OS 10.7	PASS - 2012-03-06	PASS - 2012-03-07
Ubuntu 11.10 32-bit	PASS - 2012-03-06	PASS - 2012-03-07
Ubuntu 11.10 64-bit	PASS - 2012-03-06	PASS - 2012-03-07

Issues

Issues

New

QA Investigating (QA-WANTED)

Bug	Status
bug 733709 - Lost all installed add-ons during a restart of Firefox	Henrik is investigating
bug 701404 - Silverlight crashiness using Bing Video	Vlad investigating possible regression in Silverlight
bug 716391 - Crash @je_free xul.dll@0xbb...f	Marcia investigating possible malware
bug 723444 - Angry Birds is running jerky	Paul investigating if 11.0b6 is better than 11.0b1

Engineering Investigating

Bug	Status
bug 715401 - Crash @nsCanvasRenderingContext2DAzure::ClearRect (Yahoo Toolbar)	testcase needed
bug 671468 - Crash @nsSocketOutputStream::Write (Yandex)	Honza is investigating
bug 711656 - Crash in TextStageManager::AddStagesForSubrect w/Intel GMA4500HD	Benoit Jacob is investigating
bug 731720 - Hang logging into Evernote	Alex Keybl is investigating
bug 656758 - Crash @nsSocketInputStream::Read (nsUrlClassifierDBService)	Josh Aas is investigating
bug 720679 - Crash @ mozilla::dom::workers::WorkerPrivate::CancelAllTimeouts	Gregory Szorc is investigating

Followup Needed

Beta 7

Builds: HTTPS
Litmus: testrun, report

Checklist

{{ |}} Functional automated tests, synced to Litmus
{{ |}} Manual tests
{{ |}} Bug fix verifications
{{ |}} Updates on releasetest
{{ |}} Peer sign-off
{{ |}} Updates on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows 2000	PASS - 2012-03-08
Windows XP	PASS - 2012-03-08
Windows Vista	PASS - 2012-03-08
Windows 7 32-bit	PASS - 2012-03-08
Windows 7 64-bit	PASS - 2012-03-08
Mac OS X 10.6	PASS - 2012-03-08
Mac OS X 10.7	PASS - 2012-03-08
Ubuntu 11.10 32-bit	PASS - 2012-03-08
Ubuntu 11.10 64-bit	PASS - 2012-03-08

Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoketests tests.

Platform	Assignee	Result: report
Windows 2000	Simona	PASS - 2012-03-09
Windows XP	Ioana	PASS - 2012-03-09
Windows Vista	Simona	PASS - 2012-03-09
Windows 7 32-bit	Mihaela	PASS - 2012-03-09
Windows 7 64-bit	Mihaela	PASS - 2012-03-09
Mac OS X 10.6.8	Vlad	PASS - 2012-03-09
Mac OS X 10.7.3	Vlad	PASS - 2012-03-09
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-03-09
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-03-09

Blocklist Tests

Bugs

bug 715401 – Firefox Crash @ nsCanvasRenderingContext2DAzure::ClearRect with Yahoo Toolbar
bug 711656 – Firefox 11.0a1 Crash in TextStageManager::AddStagesForSubrect with Intel GMA X4500HD, 4500MHD, and HD Graphics up to 8.15.10.2141

Summary

The current thinking is that the above two crashes are being caused by improper blocklisting of drivers. In other words, some old versions are *not* being blocklisted when indeed they should.

What to Test

Install one of the old drivers
In Firefox 10.0.2, verify blocklisted in about:support ("Direct2d Enabled" status)
In Firefox 11.0b7, verify blocklisted in about:support ("Direct2d Enabled" status)
Install Yahoo Toolbar
Try to trigger a crash by playing around with websites using canvas

NOTE: Use these try-server builds only if there are no 11.0b7-candidate builds available

Results

Driver	GPU	Blocklisted?	Crash?
Example
7.14.1.1134	VMware SVGA 3D (Microsoft Corporation - WDDM)	Fx10: YES, Fx11: NO	NO
Intel
8.15.10.1749	Blocked for your graphics card because of unresolved driver issues.	Fx10: YES, Fx11: YES	NO
8.15.10.1930	Blocked for your graphics card because of unresolved driver issues.	Fx10: YES, Fx11: YES	NO
ATI

NVidia
8.17.11.9539	0/1. Blocked for your graphics card because of unresolved driver issues. (NVidia GeForce 210)	Fx10: YES, Fx11: YES	NO
8.17.12.7533	1/1 Direct3D 10	Fx10: NO, Fx11: NO	NO

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Softvision verifications spreadsheet

Queries

QA+: non-security bugs needing verification (0 / 1 VERIFIED)
QA+ SG: security bugs which need verification (0 / 13 VERIFIED)
status-firefox11:fixed: bugs needing triage for QA+ (7 / 7 TRIAGED)
UNCO: unconfirmed bugs filed during Firefox 11 (0 / 39 TRIAGED)

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b6 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b7. This is a temporary stop-gap to ease burden on RelEng resources.

Platform	Releasetest: report	Beta: report
Windows 2000	PASS - 2012-03-09	PASS - 2012-03-09
Windows XP	PASS - 2012-03-09	PASS - 2012-03-09
Windows Vista	PASS - 2012-03-09	PASS - 2012-03-09
Windows 7 32-bit	PASS - 2012-03-09	PASS - 2012-03-09
Windows 7 64-bit	PASS - 2012-03-09	PASS - 2012-03-09
Mac OS X 10.6	PASS - 2012-03-09	PASS - 2012-03-09
Mac OS 10.7	PASS - 2012-03-09	PASS - 2012-03-09
Ubuntu 11.10 32-bit	PASS - 2012-03-09	PASS - 2012-03-09
Ubuntu 11.10 64-bit	PASS - 2012-03-09	PASS - 2012-03-09

Issues

Some failures due to slow mirrors

Issues

New

Beta 8

RESPIN FOR bug 711656

Builds: HTTPS
Litmus: testrun, report

Checklist

{{ |}} Functional automated tests, synced to Litmus
{{ |}} Manual tests
{{ |}} Bug fix verifications
{{ |}} Updates on releasetest
{{ |}} Peer sign-off
{{ |}} Updates on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows 2000	PASS - 2012-03-10
Windows XP	PASS - 2012-03-10
Windows Vista	PASS - 2012-03-10
Windows 7 32-bit	PASS - 2012-03-10
Windows 7 64-bit	PASS - 2012-03-10
Mac OS X 10.6	PASS - 2012-03-10
Mac OS X 10.7	PASS - 2012-03-10
Ubuntu 11.10 32-bit	PASS - 2012-03-10
Ubuntu 11.10 64-bit	PASS - 2012-03-10

Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoketests tests.

Platform	Assignee	Result: report
Windows 2000	ashughes	PASS - 2012-03-10
Windows XP	ashughes	PASS - 2012-03-10
Windows Vista	ashughes	PASS - 2012-03-10
Windows 7 32-bit	jsmith	PASS - 2012-03-11
Windows 7 64-bit	jsmith	PASS - 2012-03-11
Mac OS X 10.6.8	marcia	PASS - 2012-03-10
Mac OS X 10.7.3	ashughes	PASS - 2012-03-10
Ubuntu 11.10 32-bit	ashughes	PASS - 2012-03-10
Ubuntu 11.10 64-bit	ashughes	PASS - 2012-03-10

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Softvision verifications spreadsheet

Queries

QA+: non-security bugs needing verification (0 / 1 VERIFIED)
QA+ SG: security bugs which need verification (0 / 13 VERIFIED)
status-firefox11:fixed: bugs needing triage for QA+ (7 / 7 TRIAGED)
UNCO: unconfirmed bugs filed during Firefox 11 (0 / 39 TRIAGED)

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b6 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b7. This is a temporary stop-gap to ease burden on RelEng resources.

Platform	Releasetest: report	Beta: report
Windows 2000	PASS - 2012-03-11	PASS - 2012-03-11
Windows XP	PASS - 2012-03-11	PASS - 2012-03-11
Windows Vista	PASS - 2012-03-11	PASS - 2012-03-11
Windows 7 32-bit	PASS - 2012-03-11	PASS - 2012-03-11
Windows 7 64-bit	PASS - 2012-03-11	PASS - 2012-03-11
Mac OS X 10.6	PASS - 2012-03-11	PASS - 2012-03-11
Mac OS 10.7	PASS - 2012-03-11	PASS - 2012-03-11
Ubuntu 11.10 32-bit	PASS - 2012-03-11	PASS - 2012-03-11
Ubuntu 11.10 64-bit	PASS - 2012-03-11	PASS - 2012-03-11

Issues

Issues

Full Query

ID	Summary	Priority	Status
734721	Full-screen HTML 5 Video produces short random freezes when watching nature by numbers in Win 7 64-bit	--	RESOLVED
734723	Printing http://www.mozilla.org/en-US/firefox/fx/ is missing many parts of the page (color, images, etc)	--	RESOLVED
734725	Uninstalling Firefox leaves behind some folders and files in program files location where it was installed	--	RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);

RC

SCRAPPED DUE TO bug 711656, SEE RC build 2

Builds: HTTPS
Litmus: testrun, report

Checklist

{{ |}} Functional automated tests, synced to Litmus
{{ |}} Manual tests
{{ |}} Bug fix verifications
{{ |}} Updates on releasetest
{{ |}} Peer sign-off
{{ |}} Updates on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows 2000	PASS - 2012-03-09
Windows XP	PASS - 2012-03-09
Windows Vista	PASS - 2012-03-09
Windows 7 32-bit	PASS - 2012-03-09
Windows 7 64-bit	PASS - 2012-03-09
Mac OS X 10.6	PASS - 2012-03-09
Mac OS X 10.7	PASS - 2012-03-09
Ubuntu 11.10 32-bit	PASS - 2012-03-09
Ubuntu 11.10 64-bit	PASS - 2012-03-09

Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoketests tests.

Platform	Assignee	Result: report
Windows 2000	tester	result - YYYY-MM-DD
Windows XP	tester	result - YYYY-MM-DD
Windows Vista	tester	result - YYYY-MM-DD
Windows 7 32-bit	jsmith	result - YYYY-MM-DD
Windows 7 64-bit	jsmith	result - YYYY-MM-DD
Mac OS X 10.6.8	marcia	result - YYYY-MM-DD
Mac OS X 10.7.3	tester	result - YYYY-MM-DD
Ubuntu 11.10 32-bit	tester	result - YYYY-MM-DD
Ubuntu 11.10 64-bit	ashughes	result - YYYY-MM-DD

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Softvision verifications spreadsheet

Queries

QA+: non-security bugs needing verification (0 / 1 VERIFIED)
QA+ SG: security bugs which need verification (0 / 13 VERIFIED)
status-firefox11:fixed: bugs needing triage for QA+ (7 / 7 TRIAGED)
UNCO: unconfirmed bugs filed during Firefox 11 (0 / 39 TRIAGED)

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b6 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b7. This is a temporary stop-gap to ease burden on RelEng resources.

Platform	Releasetest: [ report]	Beta: [ report]
Windows 2000	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows XP	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows Vista	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 7 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 7 64-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS X 10.6	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS 10.7	result - YYYY-MM-DD	result - YYYY-MM-DD
Ubuntu 11.10 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Ubuntu 11.10 64-bit	result - YYYY-MM-DD	result - YYYY-MM-DD

Issues

Some failures due to slow mirrors

Issues

New

Bug	Status
bug 734721 Full-screen HTML 5 Video produces short random freezes

Tracking

Bug	Status
bug 715401 – Firefox Crash @ nsCanvasRenderingContext2DAzure::ClearRect with Yahoo Toolbar
bug 711656 – Firefox 11.0a1 Crash in TextStageManager::AddStagesForSubrect with Intel GMA X4500HD, 4500MHD, and HD Graphics up to 8.15.10.2141

RC2

RESPIN FOR bug 711656

Builds: HTTPS
Litmus: testrun, report

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications
[DONE] Updates on betatest
[DONE] Peer sign-off
[DONE] Updates on releasetest
[DONE] Updates on release
{{ |}} DTC testing (can only be run with release snippets)

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows 2000	PASS - 2012-03-12
Windows XP	PASS - 2012-03-12
Windows Vista	PASS - 2012-03-12
Windows 7 32-bit	PASS - 2012-03-12
Windows 7 64-bit	PASS - 2012-03-12
Mac OS X 10.6	PASS - 2012-03-12
Mac OS X 10.7	PASS - 2012-03-12
Ubuntu 11.10 32-bit	PASS - 2012-03-12
Ubuntu 11.10 64-bit	PASS - 2012-03-12

Test Failures

Manual Tests

The following are the detailed results of the manual Smoketests tests.

Platform	Assignee	Result: report
Windows 2000	Simona	PASS - 2012-03-13
Windows XP	Ioana	PASS - 2012-03-13
Windows Vista	Simona	PASS - 2012-03-13
Windows 7 32-bit	Mihaela	PASS - 2012-03-13
Windows 7 64-bit	Paul	PASS - 2012-03-13
Mac OS X 10.6.8	Vlad	PASS - 2012-03-13
Mac OS X 10.7.3	Vlad	PASS - 2012-03-13
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-03-13
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-03-13

Focus Tests

The following are the detailed results of the focused testing and running the regression tests.

Platform	Assignee	Result: report
Windows 7 32-bit	Mihaela	PASS - 2012-03-13
Mac OS X 10.6.8	VladG	PASS - 2012-03-13
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-03-13

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox11 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Softvision verifications spreadsheet

Queries

QA+: non-security bugs needing verification (0 / 1 VERIFIED)
QA+ SG: security bugs which need verification (0 / 13 VERIFIED)
status-firefox11:fixed: bugs needing triage for QA+ (7 / 7 TRIAGED)
UNCO: unconfirmed bugs filed during Firefox 11 (0 / 39 TRIAGED)

Update Tests

The following are details of update verifications

Platform	Betatest: report	Releasetest: report	Release: report
Windows 2000	PASS - 2012-03-12	PASS - 2012-03-13	PASS - 2012-03-13
Windows XP	PASS - 2012-03-12	PASS - 2012-03-13	PASS - 2012-03-13
Windows Vista	PASS - 2012-03-12	PASS - 2012-03-13	PASS - 2012-03-13
Windows 7 32-bit	PASS - 2012-03-12	PASS - 2012-03-13	PASS - 2012-03-13
Windows 7 64-bit	PASS - 2012-03-12	PASS - 2012-03-13	PASS - 2012-03-13
Mac OS X 10.6	PASS - 2012-03-12	PASS - 2012-03-13	PASS - 2012-03-13
Mac OS 10.7	PASS - 2012-03-12	PASS - 2012-03-13	PASS - 2012-03-13
Ubuntu 11.10 32-bit	PASS - 2012-03-12	PASS - 2012-03-13	PASS - 2012-03-13
Ubuntu 11.10 64-bit	PASS - 2012-03-12	PASS - 2012-03-13	PASS - 2012-03-13

Issues

Default to Compatible Addons

Pre-update Endurance Testrun (Memory) reports

Mac: 199 - 295 MB
Win XP: 127 - 270 MB
Win 7: 132 - 248 MB

Post-update Endurance Testrun (Memory) reports

Mac: 210 - 219 MB
Win XP: 99 - 148 MB
Win 7: 125 - 157 MB

Add-on Incompatibilities reports

[Addon] Яндекс.Бар 4.2.2 (disabled, incompatible)
[Addon] Text Link 4.0.2011021601 (disabled, incompatible)
[Addon] United States English Spellchecker 5.0.1 (disabled, incompatible)
[Addon] PriceGong 2.2.0 (disabled, incompatible)
[Addon] IE Tab 1.5.20090525 (disabled, incompatible)
[Addon] British English Dictionary 1.19.1 (disabled, incompatible)
[Addon] Google Toolbar for Firefox 7.1.20110512W (disabled, incompatible)
[Theme] NASA Night Launch 0.6.20120129 (disabled, compatible)
[Theme] Default 10.0.2 (disabled, compatible)

Issues

New

Bug	Status
bug 734837 - Broken history when navigating to and from cnn.com	QA investigating -- might be server-side issue
bug 735243 - On HTML5 videos Esc key exits from both full screen and Highlighter	NEW

Releases/Firefox 11/Test Plan

Contents

Firefox 11 Test Plan

Summary

Details

Features

Endurance Automation

Functional Automation

Bug Triage

Aurora Merge

Functional Automation

Updates

Beta 1

Automated Tests

Manual Tests

Update Tests

Bug Fix Verifications

Issues Found

Known Issues

Feedback

Beta 2

Automated Tests

Manual Tests

Update Tests

Bug Fix Verifications

Issues Found

Feedback

Beta 3 build 2

Automated Tests

Manual Tests

Smoketests

Focus Tests

Update Tests

Bug Fix Verifications

Issues Found

Beta 4

Automated Tests

Manual Tests

Smoketests

Focus Tests

Update Tests

Bug Fix Verifications

Issues

New

Tracking

Beta 5

Automated Tests

Manual Tests

Smoketests

Focus Tests

Update Tests

Bug Fix Verifications

Issues

Day 2 Testing

Blocklist Testing

Default to Compatible

Beta 6

Automated Tests

Manual Tests

Smoketests

Focus Tests

Bug Fix Verifications

Update Tests

Issues

Beta 7

Automated Tests

Manual Tests

Smoketests

Blocklist Tests

Bug Fix Verifications

Update Tests

Issues

Beta 8

Automated Tests

Manual Tests

Smoketests

Bug Fix Verifications

Update Tests

Issues

RC