« Firefox 12 | Firefox 13 | Firefox 14 »

Firefox 13 Test Plan

Summary

Milestone	Firefox 13
Status	Currently in Beta -- releasing on June 5, 2012
Lead	Anthony Hughes (irc: ashughes)
2nd	Juan Becerra (irc: juanb)
Builds	[ftp] [http] [web]

The following is the test plan for Firefox 13 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 13. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

2012-01-31: [DONE] Nightly Merge
2012-03-16: [DONE] Aurora Merge
2012-04-24: [DONE] Pre-Beta feature sign-offs
2012-04-26: [DONE] Beta 1
2012-05-03: [DONE] Beta 2
2012-05-10: [DONE] Beta 3
2012-05-17: [DONE] Beta 4
2012-05-24: [DONE] Beta 5
2012-05-28: [DONE] Beta 6
2012-05-31: [DONE] Beta 7
2012-06-01: {{|}} Release Candidate
2012-06-05: {{|}} Final

Nightly

Merge

Functional Automation - PASS

http://mozmill-release.blargon7.com/#/functional/reports?branch=13.0&platform=All&from=2012-01-30&to=2012-02-01

Update Spotchecks - PASS

http://mozmill-release.blargon7.com/#/update/detail?branch=13.0&channel=nightly&from=2012-01-30&to=2012-02-06&target=13.0a1

Features

See the Feature Release Tracking wiki for a general overview

[DONE]

New Tab Page (Virgil Dicu)
Home Tab (Vlad Ghetiu)
Clean up user profile (Ioana Budnar)
Enable Smooth Scrolling (Mihaela Velimiroviciu)
Automatic Session Restore with Tabs on Demand (Mihaela Velimiroviciu)
SPDY (Ioana Budnar)
Mandatory ASLR on Windows for binary components (Virgil Dicu)

[ON TRACK]

[AT RISK]

[MISSED]

URL Autocomplete (Vlad Ghetiu); delayed to Firefox 14
Enhancements to help mitigate search hijacking (Paul Silaghi); release target removed
Incremental garbage collection (Ioana Budnar); delayed to Firefox 14
Hang Detector and Reporter (Simona Badau); delayed to Firefox 14
Web Apps Integration to Desktop (Jason Smith); delayed to Firefox 14
Silent Update: Background updates (Vlad Ghetiu); delayed to Firefox 14
Speedy Session Restore; delayed to Firefox 14
Silent Update: Updated workflow for users with incompatible add-ons (Ioana Budnar); delayed to Firefox 14
Panel-based Download Manager (Simona Badau); delayed to Firefox 14
Improve display of location bar results (Virgil Dicu); target was removed.

Endurance Automation

The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

What to Do

Check the Endurance Charts
If you see an anomalous event (ie. spike or dip), get the report for the specific event
If the event happens twice in a row, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Mihaela Velimiroviciu) - Last Updated 2012-02-01

spreadsheet
bug 724553 Memory usage spike detected in Mozmill Endurance tests for Mac OS X 10.6.8 x86_64
- Caused by memory leak in Private Browsing mode: bug 724677
bug 725701 Memory usage spike detected in Mozmill Endurance tests for Win 5.1.2600 x86
bug 728871 - High memory usage spike detected in Mozmill Endurance tests on Feb 19 caused by opening new tabs
bug 734857 - Memory leak increase with about 10% in open new tabs tests (Win XP)

Functional Automation

The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

What to Do

Check the testrun reports
If you see any failing tests, check that there are not any bugs on file for it already
If not, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Vlad Maniac / Alex Lakatos) - Last Updated 2012-02-01

No issues found

Unconfirmed Bug Triage

The purpose of this section is to triage UNCONFIRMED incoming bugs. The below charts provides a summary of the unconfirmed bugs currently in the Firefox product:

P1 Task - Get UNCONFIRMED Bugs Out of Firefox General and Untriaged If Possible

Check this link for UNCONFIRMED bugs found between 4/24/2012 and 6/5/2012
Read each bug and determine what product and component they belong under in bugzilla
Move the bug to the right product and component
CC any applicable team members that may be interested in the bug

P2 Task - Confirm UNCONFIRMED Bugs in this 6-Week Cycle

Check the UNCONFIRMED bugs
Test the bugs in the latest Firefox Nightly
If you can't reproduce the bug, ask for the reporter to provide more information
If you reproduce the bug, add your testing information and mark the bug NEW
Notify Anthony Hughes of the NEW issue

P3 Task - Confirm UNCONFIRMED Bugs from Other Releases

Check the UNCONFIRMED bugs
Test the bugs in the latest Firefox Nightly
If you can't reproduce the bug, ask for the reporter to provide more information
If you reproduce the bug, add your testing information and mark the bug NEW
Notify Anthony Hughes of the NEW issue

Current Status (Vlad Ghetiu / Virgil Dicu) - Last Updated 2012-02-01

Result Spreadsheet

Aurora Merge

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows 2000	Result
Windows XP	Result
Windows Vista	Result
Windows 7 32-bit	Result
Windows 7 64-bit	Result
Mac OS X 10.6	Result
Mac OS X 10.7	Result
Ubuntu 11.04 32-bit	Result
Ubuntu 11.04 64-bit	Result

Issues Found

Updates

The following are details of update verifications for Firefox 12 merge to Aurora

Auroratest

Windows XP: PASS (2012-03-16)
- Mozilla/5.0 (Windows NT 5.1; rv:12.0a2) Gecko/20120312 Firefox/12.0a2
- Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20120316 Firefox/13.0a2
Windows 7: PASS (2012-03-16)
- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0a2) Gecko/20120309 Firefox/12.0a2
- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120316 Firefox/13.0a2
Mac OS X 10.6:
- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:12.0a2) Gecko/20120309 Firefox/12.0a2
- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20120316 Firefox/13.0a2
Ubuntu 11.10: PASS (2012-03-16)
- Mozilla/5.0 (X11; Linux x86_64; rv:12.0a2) Gecko/20120310 Firefox/12.0a2
- Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20120316 Firefox/13.0a2

Aurora

[ON TRACK] [ashughes] Windows XP:
[ON TRACK] [ashughes] Windows 7:
[ON TRACK] [ashughes] Mac OS X 10.6:
[ON TRACK] [ashughes] Ubuntu 11.04:

Issues Found

Manual Spot Checks

1. New Home Page exploratory (bug 711157, bug 731546)

2. UAC Service exploratory

Aurora Feature Sign-off

The tests for the features with Litmus testcases can be run here.

Mandatory ASLR on Windows for binary components (Virgil Dicu)
- SIGNED OFF on Aurora - 2012-03-19
Silent Update: Background updates (Vlad Ghetiu)
- Feature didn't land in Firefox 13
New Tab Page (Virgil Dicu)
- SIGNED OFF on Aurora - 2012-03-16
Home Tab (Vlad Ghetiu)
- Phase 1 - Landed - New about:home page - SIGNED OFF - 2012-03-19 testcases
- Phase 2 - Still in development - feature page
Clean up user profile (Ioana Budnar)
- UI from safe mode dialog and installer not implemented yet. UI from about:support landed in Firefox 13.0.
- SIGNED OFF - 2012-03-19
Hang Detector and Reporter (Simona Badau)
- Testing end-to-end is not possible until the Symbolication Server will be publicly accessible (currently is hosted on a VM - Bug 721743).
Enable Smooth Scrolling (Mihaela Velimiroviciu)
- SIGNED OFF - 2012-03-19
Incremental garbage collection (Ioana Budnar)
- SIGNED OFF - 2012-03-19
Automatic Session Restore with Tabs on Demand (Mihaela Velimiroviciu)
- SIGNED OFF - 2012-03-16
URL Autocomplete (Vlad Ghetiu)
- SIGNED OFF - 2012-03-19 testcases

QAWANTED

Bugs: query

Bug	Status
bug 715149 Assertion failure: [infer failure] Missing type pushed 0: void	Trouble tracking regression (2012-03-26)
bug 738392 Plugin doesn't render inside a CSS transform on Mac	Not reproducible (2012-03-26)

Beta 1

Builds: FTP, HTTP
Litmus: testrun, report

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications
{{ |}} Check with Marcia for top-crashers spiking
[DONE] Update tests on releasetest
{{ |}} Get final sign-off from Bob Moss
{{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows XP	PASS - 2012-04-25
Windows Vista	PASS - 2012-04-25
Windows 7 32-bit	PASS - 2012-04-25
Windows 7 64-bit	PASS - 2012-04-25
Mac OS X 10.6	PASS - 2012-04-25
Mac OS X 10.7	PASS - 2012-04-25
Ubuntu 11.10 32-bit	PASS - 2012-04-25
Ubuntu 11.10 64-bit	PASS - 2012-04-25

Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the Firefox 13.0 Smoketests in Litmus and report results below.

Platform	Assignee	Result
Windows XP	Ioana	PASS - 2012-04-26
Windows Vista	Simona	PASS - 2012-04-26
Windows 7 32-bit	Simona	PASS - 2012-04-26
Windows 7 64-bit	Virgil	PASS - 2012-04-26
Mac OS X 10.6.8	Mihaela	PASS - 2012-04-26
Mac OS X 10.7.3	Mihaela	PASS - 2012-04-26
Ubuntu 11.10 32-bit	Virgil	PASS - 2012-04-26
Ubuntu 11.10 64-bit	Ioana	PASS - 2012-04-26

Results in Litmus

Regression Testing

Run the Firefox 13.0 Regression Tests and Firefox 13.0 Web App Compatibility in Litmus and report the results below.

Platform	Assignee	Result: report
Windows 7 32-bit	Virgil	PASS - 2012-04-26
Mac OS X 10.6.8	Mihaela	PASS - 2012-04-26
Ubuntu 11.10 32-bit	Paul	PASS - 2012-04-26
Windows XP	Ioana	PASS - 2012-04-26

Feature Testing

Run the Firefox 13.0 Features Sign-off tests in Litmus and report the results below. Additionally do some exploratory testing to make sure no regressions have been introduced to related legacy features.

Feature	Owner	Win32	Mac	Linux
Mandatory ASLR on Windows for binary components	Virgil Dicu	PASS - 2012-04-26	n/a	n/a
New Tab Page	Virgil Dicu	PASS - 2012-04-26	PASS - 2012-04-26	PASS - 2012-04-26
Home Tab	Vlad Ghetiu	PASS - 2012-04-26	PASS - 2012-04-26	PASS - 2012-04-26
Clean up user profile	Ioana Budnar	PASS - 2012-04-26	PASS - 2012-04-26	PASS - 2012-04-26
Enable Smooth Scrolling	Mihaela Velimiroviciu	PASS - 2012-04-26	PASS - 2012-04-26	PASS - 2012-04-26
Automatic Session Restore with Tabs on Demand	Mihaela Velimiroviciu	PASS - 2012-04-26	PASS - 2012-04-26	PASS - 2012-04-26

Update Tests

The following are details of update verifications

Platform	Releasetest: report	Beta: report
Windows XP	PASS - 2012-04-26	PASS - 2012-04-26
Windows Vista	PASS - 2012-04-26	PASS - 2012-04-26
Windows 7 32-bit	PASS - 2012-04-26	PASS - 2012-04-26
Windows 7 64-bit	PASS - 2012-04-26	PASS - 2012-04-26
Mac OS X 10.6	PASS - 2012-04-26	PASS - 2012-04-26
Mac OS 10.7	PASS - 2012-04-26	PASS - 2012-04-26
Ubuntu 11.10 32-bit	PASS - 2012-04-26	PASS - 2012-04-26
Ubuntu 11.10 64-bit	PASS - 2012-04-26	PASS - 2012-04-26

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox13 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]

Query	Triaged	Remaining
QA+: for testers to verify functional fixes	7	69
QA+ SG: for the Security team to verify security fixes	0	23
status-firefox13:fixed: for QA lead to triage for potential QA+ bugs	201	0

Softvision verifications etherpad

QAWANTED

Bug (query)	Status
bug 747847 crash in nsJSContext::EvaluateString with Skype extension 5.11	Investigating reproducible testcase for Firefox 12
bug 744063 'Fullscreen playback' NVIDIA stereo 3d for youtube not working in latest Aurora v13	Investigating regression range in Firefox 13 Nightly (2012-04-26)
bug 733337 Picture being rendered outside intended region	RESOLVED FIXED (2012-04-26)

Issues Found

Full Query

ID	Summary	Priority	Status
749112	"Live Bookmarks Loading" message is displayed at the bottom of the RSS feeds	--	RESOLVED
749115	"Live Bookmark loading..." message does not disappear after RSS list is refreshed	--	RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);

Known Issues

Bugzilla query error

error, http-bad-status, Array

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: FTP, HTTP

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 2


Builds	FTP, HTTP
ID	20120501201020
Changeset	168f3814b32e

Checklist

[DONE] Functional automated tests, synced to Litmus
[ON TRACK] Manual tests
[ON TRACK] Bug fix verifications
[DONE] Update tests on releasetest
[DONE] Get final sign-off from Bob Moss
{{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows XP	PASS - 2012-05-02
Windows Vista	PASS - 2012-05-02
Windows 7 32-bit	PASS - 2012-05-02
Windows 7 64-bit	PASS - 2012-05-02
Mac OS X 10.6	PASS - 2012-05-02
Mac OS X 10.7	PASS - 2012-05-02
Ubuntu 11.10 32-bit	PASS - 2012-05-02
Ubuntu 11.10 64-bit	PASS - 2012-05-02

Test Failures (known bugs)

/testAddons/testManagerKeyboardShortcut.js
/testAddons/testPluginDisableAffectsAboutPlugins.js (bug 739935)
/testAwesomeBar/testAccessLocationBar.js

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the Firefox 13.0 Smoketests in Litmus and report results below.

Platform	Assignee	Result
Windows XP	Ioana	PASS - 2012-05-02
Windows Vista	Simona	PASS - 2012-05-03
Windows 7 32-bit	Mihaela	PASS - 2012-05-02
Windows 7 64-bit	Virgil	PASS - 2012-05-02
Mac OS X 10.6.8	Mihaela	PASS - 2012-05-02
Mac OS X 10.7.3	Ioana	PASS - 2012-05-02
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-05-02
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-05-02

Results in Litmus

Regression Testing

Run the Firefox 13.0 Regression Tests and Firefox 13.0 Web App Compatibility in Litmus and report the results below.

Platform	Assignee	Result: report
Windows 7 32-bit	Paul	PASS - 2012-05-03
Mac OS X 10.6.8	Mihaela	PASS - 2012-05-03
Ubuntu 11.10 32-bit	Virgil	PASS - 2012-05-03
Windows XP	Ioana	PASS - 2012-05-03

Feature Testing

Run the Firefox 13.0 Features Sign-off tests in Litmus and report the results below. Additionally do some exploratory testing to make sure no regressions have been introduced to related legacy features.

Feature	Owner	Win32	Mac	Linux
Mandatory ASLR on Windows for binary components	Virgil Dicu	PASS - 2012-05-04	n/a	n/a
New Tab Page	Virgil Dicu	PASS - 2012-05-03	PASS - 2012-05-03	PASS - 2012-05-03
Home Tab	Vlad Ghetiu	PASS - 2012-05-03	PASS - 2012-05-03	PASS - 2012-05-03
Clean up user profile	Ioana Budnar	PASS - 2012-05-03	PASS - 2012-05-03	PASS - 2012-05-03
Enable Smooth Scrolling	Mihaela Velimiroviciu	result - yyyy-mm-dd	PASS - 2012-05-03	result - yyyy-mm-dd
Automatic Session Restore with Tabs on Demand	Mihaela Velimiroviciu	PASS - 2012-05-03	PASS - 2012-05-03	result - yyyy-mm-dd

Update Tests

The following are details of update verifications

Platform	Releasetest: report	Beta: report
Windows XP	PASS - 2012-05-02	PASS - 2012-05-03
Windows Vista	PASS - 2012-05-02	PASS - 2012-05-03
Windows 7 32-bit	PASS - 2012-05-02	PASS - 2012-05-03
Windows 7 64-bit	PASS - 2012-05-02	PASS - 2012-05-03
Mac OS X 10.6	PASS - 2012-05-02	PASS - 2012-05-03
Mac OS 10.7	PASS - 2012-05-02	PASS - 2012-05-03
Ubuntu 11.10 32-bit	PASS - 2012-05-02	PASS - 2012-05-03
Ubuntu 11.10 64-bit	PASS - 2012-05-02	PASS - 2012-05-03

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox13 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]

Query	Triaged	Remaining
QA+: for testers to verify functional fixes	5	69
QA+ SG: for the Security team to verify security fixes	0	23
status-firefox13:fixed: for QA lead to triage for potential QA+ bugs	201	0

Softvision verifications etherpad

QAWANTED

Bug (query)	Status
bug 700288 Crash in mozilla::gfx::BaseRect::UnionEdges with AMD Radeon HD 6xxx series	NEW - Feedback Needed - Unreproducible (2012-05-08)
bug 729507 Crash in JSObject::finish	NEW - Feedback Needed - Unreproducible (2012-05-08)
bug 750625 Crash in nsUrlClassifierPrefixSet::GetPrefixes	NEW - Feedback needed - Where to Begin (2012-05-08)
bug 736152 "Good Primes" web worker test hangs browser and is slow to update	NEW - Need to investigate (2012-05-08)
bug 744063 'Fullscreen playback' NVIDIA stereo 3d for youtube not working in latest Aurora v13	NEW - jsmith investigating (2012-05-08)
bug 751585 XPCOM component loading rejected due to ASLR not set	FIXED - ashughes testing fix (2012-05-08)
bug 751753 Firefox 12 aggressively caching pages	NEW - juanb investigating (2012-05-08)

Issues Found

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Known Issues

Full Query

ID	Summary	Priority	Status
709029	The RealPlayer plugin crashes when trying to play a RealPlayer video	--	RESOLVED
725296	Browser is unresponsive while Silverlight content is loading	--	RESOLVED
725614	Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content	P5	RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: FTP, HTTP

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 3


Builds	FTP, HTTP
ID	20120508180640
Changeset	f40f8e40bae8

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications (target 50%)
[MISSED] Security fix verifications (target 25%)
[DONE] Update tests on releasetest
[DONE] Get final sign-off from Bob Moss
{{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows XP	PASS - 2012-05-09
Windows Vista	PASS - 2012-05-09
Windows 7 32-bit	PASS - 2012-05-09
Windows 7 64-bit	PASS - 2012-05-09
Mac OS X 10.6	PASS - 2012-05-09
Mac OS X 10.7	PASS - 2012-05-09
Ubuntu 11.10 32-bit	PASS - 2012-05-09
Ubuntu 11.10 64-bit	PASS - 2012-05-09

Test Failures (known bugs)

/testPasswordManager/testPasswordNotificationBar.js (bug 749156)
/testPasswordManager/testPasswordSavedAndDeleted.js (bug 749206)
/testPrivateBrowsing/testCloseWindow.js (bug 573461)
/testPrivateBrowsing/testDownloadManagerClosed.js (bug 573461)

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the Firefox 13.0 Smoketests in Litmus and report results below.

Platform	Assignee	Result
Windows XP	Ioana	PASS - 2012-05-10
Windows Vista	Simona	PASS - 2012-05-10
Windows 7 32-bit	Mihaela	PASS - 2012-05-10
Windows 7 64-bit	Paul	PASS - 2012-05-10
Mac OS X 10.6.8	VladG	PASS - 2012-05-10
Mac OS X 10.7.3	VladG	PASS - 2012-05-10
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-05-09
Ubuntu 11.10 64-bit	Virgil	PASS - 2012-05-10

Results in Litmus

Regression Testing

Run the Firefox 13.0 Regression Tests and Firefox 13.0 Web App Compatibility in Litmus and report the results below.

Platform	Assignee	Result: report
Windows 7 32-bit	Mihaela	PASS - 2012-05-10
Mac OS X 10.6.8	VladG	PASS - 2012-05-10
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-05-09
Windows XP	Ioana	PASS - 2012-05-10

[DONE] Exploratory testing Etherpad

Feature Testing

Run the Firefox 13.0 Features Sign-off tests in Litmus and report the results below. Additionally do some exploratory testing to make sure no regressions have been introduced to related legacy features.

Feature	Owner	Win32	Mac	Linux
Mandatory ASLR on Windows for binary components	Virgil Dicu	PASS - 2012-05-10	n/a	n/a
New Tab Page	Virgil Dicu	PASS - 2012-05-10	PASS - 2012-05-09	PASS - 2012-05-09
Home Tab	Vlad Ghetiu	PASS - 2012-05-10	PASS - 2012-05-09	PASS - 2012-05-09
Clean up user profile	Ioana Budnar	PASS - 2012-05-10	PASS - 2012-05-09	PASS - 2012-05-09
Enable Smooth Scrolling	Mihaela Velimiroviciu	PASS - 2012-05-10	PASS - 2012-05-09	PASS - 2012-05-09
Automatic Session Restore with Tabs on Demand	Mihaela Velimiroviciu	PASS - 2012-05-10	PASS - 2012-05-09	PASS - 2012-05-09

Update Tests

The following are details of update verifications

Platform	Releasetest: report	Beta: report
Windows XP	PASS - 2012-05-10	PASS - 2012-05-11
Windows Vista	PASS - 2012-05-10	PASS - 2012-05-11
Windows 7 32-bit	PASS - 2012-05-10	PASS - 2012-05-11
Windows 7 64-bit	PASS - 2012-05-10	PASS - 2012-05-11
Mac OS X 10.6	PASS - 2012-05-10	PASS - 2012-05-11
Mac OS 10.7	PASS - 2012-05-10	PASS - 2012-05-11
Ubuntu 11.10 32-bit	PASS - 2012-05-10	PASS - 2012-05-11
Ubuntu 11.10 64-bit	PASS - 2012-05-10	PASS - 2012-05-11

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox13 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]

Query	Triaged	Remaining	% Complete
QA+: for testers to verify functional fixes	42	43	49%
QA+ SG: for the Security team to verify security fixes	0	30	0%
status-firefox13:fixed: for QA lead to triage for potential QA+ bugs	297	0	100%

Softvision verifications etherpad

QAWANTED

QAWANTED Bugs

bug 736731 Hotmail no longer auto-updates (due to "globalStorage" removal)
- Confirmed regression in Firefox 13.0b3
bug 736152 "Good Primes" web worker test hangs browser and is slow to update
- Slow Script warning in Fx10, 14, and 15
- UI responsiveness issue in Fx10, 11, 12
- Counter is "choppy" in Fx13, starts out choppy in Fx14 and 15 but improves
- Conditions are 100% reproducible
bug 744063 'Fullscreen playback' NVIDIA stereo 3d for youtube not working in latest Aurora v13
- Hardware set up, confirmed "fullscreen side-by-side" started in Fx13, working on regression range
- Side-by-side effect occurs as early as Fx12 for the progress bar hover -- is there a bug on this?
bug 751585 XPCOM component loading rejected due to ASLR not set
- Using CFF Explorer, firefox.exe appears to be using ASLR, shlwapi.dll is not -- need further instructions
bug 751753 Firefox 12 aggressively caching pages
- Fixed Nightly needs testing by reporters -- not sure if there's much QA can do to help further

Issues Found

Full Query

ID	Summary	Priority	Status
753680	Cannot enable all personas from Add-ons Manager list	--	VERIFIED
753684	Removing an add-on installed twice doesn't actually remove it	--	RESOLVED
753770	Printing progress bar is not properly displayed	--	RESOLVED
753889	stage.mo -> releases.mo breaks staging beta builds for update testing	--	VERIFIED

4 Total; 0 Open (0%); 2 Resolved (50%); 2 Verified (50%);

Known Issues

Full Query

ID	Summary	Priority	Status
585971	autocomplete matches unrelated hosts when searching for www.example.com	--	RESOLVED
709029	The RealPlayer plugin crashes when trying to play a RealPlayer video	--	RESOLVED
725296	Browser is unresponsive while Silverlight content is loading	--	RESOLVED
725614	Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content	P5	RESOLVED
726943	Fullscreen button in HTML video makes the browser go fullscreen but not the video	--	VERIFIED
727702	Undoing applying a Persona reverts back to default theme instead of old Persona	--	VERIFIED
730829	Restore "Open all in tabs" to the bottom of livemarks popups (was: "Open all in tabs" missing after bug 613588)	--	RESOLVED
749112	"Live Bookmarks Loading" message is displayed at the bottom of the RSS feeds	--	RESOLVED

8 Total; 0 Open (0%); 6 Resolved (75%); 2 Verified (25%);

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: FTP, HTTP

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 4


Builds	FTP, HTTP
ID	20120516113045
Changeset	eb5336a48ab0

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[ON TRACK] Bug fix verifications (target 50%)
[ON TRACK] Security fix verifications (target 25%)
[DONE] Update tests on releasetest
[DONE] Get final sign-off from Bob Moss
{{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows XP	PASS - 2012-05-16
Windows Vista	PASS - 2012-05-16
Windows 7 32-bit	PASS - 2012-05-16
Windows 7 64-bit	PASS - 2012-05-16
Mac OS X 10.6	PASS - 2012-05-16
Mac OS X 10.7	PASS - 2012-05-16
Ubuntu 11.10 32-bit	PASS - 2012-05-16
Ubuntu 11.10 64-bit	PASS - 2012-05-16

Test Failures (known bugs)

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the Firefox 13.0 Smoketests in Litmus and report results below.

Platform	Assignee	Result
Windows XP	Ioana	PASS - 2012-05-17
Windows Vista	Simona	PASS - 2012-05-17
Windows 7 32-bit	Mihaela	PASS - 2012-05-17
Windows 7 64-bit	Paul	PASS - 2012-05-17
Mac OS X 10.6.8	VladG	PASS - 2012-05-17
Mac OS X 10.7.4	VladG	PASS - 2012-05-17
Ubuntu 12.04 32-bit	Virgil	PASS - 2012-05-17
Ubuntu 11.10 64-bit	Ioana	PASS - 2012-05-17

Results in Litmus

Regression Testing

Run the Firefox 13.0 Regression Tests and Firefox 13.0 Web App Compatibility in Litmus and report the results below.

Platform	Assignee	Result: report
Windows 7 32-bit	Paul	PASS - 2012-05-17
Mac OS X 10.6.8	VladG	PASS - 2012-05-18
Ubuntu 11.10 64-bit	Mihaela	PASS - 2012-05-17
Windows XP	Simona	PASS - 2012-05-17

[DONE] Exploratory testing Etherpad
[DONE] Plugins testing Etherpad

Feature Testing

Run the Firefox 13.0 Features Sign-off tests in Litmus and report the results below. Additionally do some exploratory testing to make sure no regressions have been introduced to related legacy features.

Feature	Owner	Win32	Mac	Linux
Mandatory ASLR on Windows for binary components	Virgil Dicu	PASS - 2012-05-17	n/a	n/a
New Tab Page	Virgil Dicu	PASS - 2012-05-17	PASS - 2012-05-17	PASS - 2012-05-17
Home Tab	Vlad Ghetiu	PASS - 2012-05-17	PASS - 2012-05-17	PASS - 2012-05-17
Clean up user profile	Ioana Budnar	PASS - 2012-05-17	PASS - 2012-05-17	PASS - 2012-05-17
Enable Smooth Scrolling	Mihaela Velimiroviciu	PASS - 2012-05-17	PASS - 2012-05-17	PASS - 2012-05-17
Automatic Session Restore with Tabs on Demand	Mihaela Velimiroviciu	PASS - 2012-05-17	PASS - 2012-05-17	PASS - 2012-05-17

Update Tests

The following are details of update verifications

Platform	Releasetest: report	Beta: report
Windows XP	PASS - 2012-05-18	PASS - 2012-05-18
Windows Vista	PASS - 2012-05-18	PASS - 2012-05-18
Windows 7 32-bit	PASS - 2012-05-18	PASS - 2012-05-18
Windows 7 64-bit	PASS - 2012-05-18	PASS - 2012-05-18
Mac OS X 10.6	PASS - 2012-05-18	PASS - 2012-05-18
Mac OS 10.7	PASS - 2012-05-18	PASS - 2012-05-18
Ubuntu 11.10 32-bit	PASS - 2012-05-18	PASS - 2012-05-18
Ubuntu 11.10 64-bit	PASS - 2012-05-18	PASS - 2012-05-18

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox13 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]

Query	Triaged	Remaining	% Complete
QA+: for testers to verify functional fixes	51	34	60%
QA+ SG: for the Security team to verify security fixes	7	23	23%
status-firefox13:fixed: for QA lead to triage for potential QA+ bugs	297	0	100%

Softvision verifications etherpad

QAWANTED

QAWANTED Bugs

[AT RISK] bug 749010 Perma-orange Linux64 test_memoryReporters.xul crash above mozilla::dom::workers::WorkerPrivate::BlockAndCollectRuntimeStats on Beta 13
- Requires a Linux64-opt build -- not sure where to get that (2012-05-15)
[ON TRACK] bug 755974 Block AMD Radeon HD 6290/6300/6310/6320 for Direct3D 9
- Marcia is testing the staged block (2012-05-17)

Issues Found

Full Query

ID	Summary	Priority	Status
756080	The Bookmarks toolbar is set as visible automatically when resetting Firefox with at least 5 bookmarks toolbar items	--	RESOLVED
756111	Emoticons are not displayed on the Facebook chat	--	RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);

Known Issues

Full Query

ID	Summary	Priority	Status
585971	autocomplete matches unrelated hosts when searching for www.example.com	--	RESOLVED
667399	RealPlayer G2 LiveConnect plugin crash	--	RESOLVED
692446	Firefox prompts to install windows media player plugin even though it is already installed	--	RESOLVED
725296	Browser is unresponsive while Silverlight content is loading	--	RESOLVED
725614	Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content	P5	RESOLVED
726943	Fullscreen button in HTML video makes the browser go fullscreen but not the video	--	VERIFIED
727702	Undoing applying a Persona reverts back to default theme instead of old Persona	--	VERIFIED
730829	Restore "Open all in tabs" to the bottom of livemarks popups (was: "Open all in tabs" missing after bug 613588)	--	RESOLVED
749112	"Live Bookmarks Loading" message is displayed at the bottom of the RSS feeds	--	RESOLVED
753680	Cannot enable all personas from Add-ons Manager list	--	VERIFIED
753684	Removing an add-on installed twice doesn't actually remove it	--	RESOLVED
753770	Printing progress bar is not properly displayed	--	RESOLVED
753889	stage.mo -> releases.mo breaks staging beta builds for update testing	--	VERIFIED
754829	In Full Screen mode - YouTube videos player changes dimmension	--	RESOLVED

14 Total; 0 Open (0%); 10 Resolved (71.43%); 4 Verified (28.57%);

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: FTP, HTTP

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 5


Builds	FTP, HTTP
ID	20120523114940
Changeset	5de6e4bd8ede

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications (target 70%)
[DONE] Security fix verifications (target 50%)
[DONE] Update tests on releasetest
{{ |}} Get stability sign-off (marcia)
{{ |}} Get final sign-off (bmoss)
{{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform	Result: report
Windows XP	PASS - 2012-05-23
Windows Vista	PASS - 2012-05-23
Windows 7 32-bit	PASS - 2012-05-23
Windows 7 64-bit	PASS - 2012-05-23
Mac OS X 10.6	PASS - 2012-05-23
Mac OS X 10.7	PASS - 2012-05-23
Ubuntu 11.10 32-bit	PASS - 2012-05-23
Ubuntu 11.10 -bit	PASS - 2012-05-23

Test Failures (known bugs)

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the Firefox 13.0 Smoketests in Litmus and report results below.

Platform	Assignee	Result
Windows XP	Ioana	PASS - 2012-05-24
Windows Vista	Simona	PASS - 2012-05-24
Windows 7 32-bit	Mihaela	PASS - 2012-05-24
Windows 7 64-bit	Paul	PASS - 2012-05-24
Mac OS X 10.6.8	VladG	PASS - 2012-05-24
Mac OS X 10.7.4	VladG	PASS - 2012-05-24
Ubuntu 12.04 32-bit	Virgil	PASS - 2012-05-24
Ubuntu 11.10 64-bit	Ioana	PASS - 2012-05-24

Results in Litmus

Regression Testing

Run the Firefox 13.0 Regression Tests and Firefox 13.0 Web App Compatibility in Litmus and report the results below.

Platform	Assignee	Result: report
Windows 7 32-bit	Mihaela	PASS - 2012-05-24
Mac OS X 10.6.8	VladG	PASS - 2012-05-24
Ubuntu 11.10 32-bit	Virgil	PASS - 2012-05-24
Windows XP	Ioana	PASS - 2012-05-24

[DONE] Exploratory testing Etherpad
[DONE] Plugins testing Etherpad

Feature Testing

Run the Firefox 13.0 Features Sign-off tests in Litmus and report the results below. Additionally do some exploratory testing to make sure no regressions have been introduced to related legacy features.

Feature	Owner	Win32	Mac	Linux
Mandatory ASLR on Windows for binary components	Virgil Dicu	result - YYYY-MM-DD	n/a	n/a
New Tab Page	Virgil Dicu	result - YYYY-MM-DD	result - YYYY-MM-DD	result - YYYY-MM-DD
Home Tab	Vlad Ghetiu	result - YYYY-MM-DD	result - YYYY-MM-DD	result - YYYY-MM-DD
Clean up user profile	Ioana Budnar	result - YYYY-MM-DD	result - YYYY-MM-DD	result - YYYY-MM-DD
Enable Smooth Scrolling	Mihaela Velimiroviciu	result - YYYY-MM-DD	result - YYYY-MM-DD	result - YYYY-MM-DD
Automatic Session Restore with Tabs on Demand	Mihaela Velimiroviciu	result - YYYY-MM-DD	result - YYYY-MM-DD	result - YYYY-MM-DD

Update Tests

The following are details of update verifications

Platform	Releasetest: report	Beta: report
Windows XP	PASS - 2012-05-24	PASS - 2012-05-24
Windows Vista	PASS - 2012-05-24	PASS - 2012-05-24
Windows 7 32-bit	PASS - 2012-05-24	PASS - 2012-05-24
Windows 7 64-bit	PASS - 2012-05-24	PASS - 2012-05-24
Mac OS X 10.6	PASS - 2012-05-24	PASS - 2012-05-24
Mac OS 10.7	PASS - 2012-05-24	PASS - 2012-05-24
Ubuntu 11.10 32-bit	PASS - 2012-05-24	PASS - 2012-05-24
Ubuntu 11.10 64-bit	PASS - 2012-05-24	PASS - 2012-05-24

Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox13 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]

Query	Triaged	Remaining	% Complete
QA+: for testers to verify functional fixes	65	23	74%
QA+ SG: for the Security team to verify security fixes	22	8	73%
status-firefox13:fixed: for QA lead to triage for potential QA+ bugs	297	0	100%

Softvision verifications etherpad

QAWANTED

QAWANTED Bugs

[AT RISK] bug 753251 Crash in nsGfxScrollFrameInner::ScrollToImpl
- QA testing dead-ended, need more leads
[ON TRACK] bug 756289 Netflix video errors "you are viewing from more than one tab"
- QA testing unsuccessful, reached out to a bug commenter
[ON TRACK] bug 757284 Issues with (large?) uploads stopping halfway (youtube, mediafire)
- Bisect found, is QAWANTED still needed?
[ON TRACK] bug 719117 Crash @ mozalloc_abort | NS_DebugBreak_P | nsIFrame::GetOffsetToCrossDoc
- Reproduced, what next?
[ON TRACK] bug 640904 Crash in JSAutoEnterCompartment::enter
- QA testing unsuccessful with Blekko toolbar, any other leads?

Issues Found

Full Query

ID	Summary	Priority	Status
756080	The Bookmarks toolbar is set as visible automatically when resetting Firefox with at least 5 bookmarks toolbar items	--	RESOLVED
756111	Emoticons are not displayed on the Facebook chat	--	RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);

Known Issues

Full Query

ID	Summary	Priority	Status
585971	autocomplete matches unrelated hosts when searching for www.example.com	--	RESOLVED
667399	RealPlayer G2 LiveConnect plugin crash	--	RESOLVED
692446	Firefox prompts to install windows media player plugin even though it is already installed	--	RESOLVED
725296	Browser is unresponsive while Silverlight content is loading	--	RESOLVED
725614	Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content	P5	RESOLVED
726943	Fullscreen button in HTML video makes the browser go fullscreen but not the video	--	VERIFIED
727702	Undoing applying a Persona reverts back to default theme instead of old Persona	--	VERIFIED
730829	Restore "Open all in tabs" to the bottom of livemarks popups (was: "Open all in tabs" missing after bug 613588)	--	RESOLVED
749112	"Live Bookmarks Loading" message is displayed at the bottom of the RSS feeds	--	RESOLVED
753680	Cannot enable all personas from Add-ons Manager list	--	VERIFIED
753684	Removing an add-on installed twice doesn't actually remove it	--	RESOLVED
753770	Printing progress bar is not properly displayed	--	RESOLVED
753889	stage.mo -> releases.mo breaks staging beta builds for update testing	--	VERIFIED
754829	In Full Screen mode - YouTube videos player changes dimmension	--	RESOLVED

14 Total; 0 Open (0%); 10 Resolved (71.43%); 4 Verified (28.57%);

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: FTP, HTTP

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 6


Builds	FTP, HTTP
ID	20120528154913
Changeset	5dd4dde1d4eb

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[ON TRACK] Bug fix verifications (target 100%)
[ON TRACK] Security fix verifications (target 100%)
[DONE] Update tests on releasetest
{{ |}} Get stability sign-off (marcia)
{{ |}} Get final sign-off (bmoss)
[DONE] Update tests on beta

Automated Tests

Results of the Mozmill automation testruns.

Functional Tests

Platform	Result: report	Failures: known bugs
Windows XP	PASS - 2012-05-29	0 - no fails
Windows Vista	PASS - 2012-05-29	0 - no fails
Windows 7 32-bit	PASS - 2012-05-29	0 - no fails
Windows 7 64-bit	PASS - 2012-05-29	0 - no fails
Mac OS X 10.6	PASS - 2012-05-29	0 - no fails
Mac OS X 10.7	PASS - 2012-05-29	0 - no fails
Ubuntu 11.10 32-bit	PASS - 2012-05-29	0 - no fails
Ubuntu 11.10 -bit	PASS - 2012-05-29	0 - no fails

Endurance Tests

Platform	Result: report	Max Memory
Windows Vista	PASS - 2012-05-29	78 MB
Windows 7 32-bit	PASS - 2012-05-29	90 MB
Windows 7 64-bit	PASS - 2012-05-29	76 MB
Mac OS X 10.6	PASS - 2012-05-29	151 MB
Mac OS X 10.7	PASS - 2012-05-29	140 MB
Ubuntu 11.10 32-bit	PASS - 2012-05-29	59 MB
Ubuntu 11.10 -bit	PASS - 2012-05-29	96 MB

Update Tests

Platform	Releasetest: report	Beta: report
Windows XP	PASS - 2012-05-29	PASS - 2012-05-29
Windows Vista	PASS - 2012-05-29	PASS - 2012-05-29
Windows 7 32-bit	PASS - 2012-05-29	PASS - 2012-05-29
Windows 7 64-bit	PASS - 2012-05-29	PASS - 2012-05-29
Mac OS X 10.6	PASS - 2012-05-29	PASS - 2012-05-29
Mac OS X 10.7	PASS - 2012-05-29	PASS - 2012-05-29
Ubuntu 11.10 32-bit	PASS - 2012-05-29	PASS - 2012-05-29
Ubuntu 11.10 -bit	PASS - 2012-05-29	PASS - 2012-05-29

Smoketests

Run the Firefox 13.0 Smoketests in Litmus and report results below.

Platform	Assignee	Result: report
Windows XP	Ioana	PASS - 2012-05-29
Windows Vista	Simona	PASS - 2012-05-29
Windows 7 32-bit	Mihaela	PASS - 2012-05-29
Windows 7 64-bit	Paul	PASS - 2012-05-29
Mac OS X 10.6.8	VladG	PASS - 2012-05-29
Mac OS X 10.7.4	VladG	PASS - 2012-05-29
Ubuntu 12.04 32-bit	Virgil	PASS - 2012-05-29
Ubuntu 11.10 64-bit	Ioana	PASS - 2012-05-29

Focus Testing

Focus Area	Win32	Mac	Linux
Special Case: YouTube Caching	PASS - 2012-05-29	PASS - 2012-05-30	PASS - 2012-05-29
Printing	FAIL - 2012-05-29	PASS - 2012-05-29	PASS - 2012-05-29
Plug-ins	FAIL - 2012-05-29	PASS - 2012-05-29	PASS - 2012-05-29
Top-sites	PASS - 2012-05-30	PASS - 2012-05-30	PASS - 2012-05-30
Mac OSX 10.8	N/A	PASS - 2012-05-30	N/A
Windows 8	FAIL - 2012-05-30	N/A	N/A

Special Case: YouTube Caching

Test: bug 757262
- Install Flash 11.2 and Firefox Beta, and open 2 windows
- First window, watch an hour long YouTube video (or longer), and let the video finish
- Once video is done, switch to the second window and do your usual browsing for another hour
- After an hour, switch back to the first window and load any non-video website in the tab you were watching the video
Anticipated Result: audio from the video plays back unexpectedly for a few seconds
Results:
- Windows 7: PASS (Mihaela)
- Ubuntu 12.04: PASS (Virgil)
- Mac OSX 10.6: PASS (VladG)
Details:

Printing

Tests:
- Print previews
- Print to file
- PDFs, Flash content, SVG content
- Rich websites (especially Flash content re: bug 747055)
- keep an eye out for crashes
Results:
- Windows 7: FAIL (Mihaela)
  - See bugs bellow
- Ubuntu 12.04: PASS (Paul)
- Mac OSX 10.6: PASS (VladG)
  - There is no "print to file" option on Mac, just save as pdf
Details & Bugs:
- bug 759334 - Print to file creates a file with unknown format
- bug 134002 - Content of plugins (flash, java applets) doesn't get rendered for printing or print preview
- bug 184537 - Print preview doesn't dynamically update page number after scrolling or adjusting settings (doesn't reflect page in view)

Plug-ins

Tests:
- Flash (11.2, 11.3)
- Java
- Silverlight
- QuickTime
- Try videos, games, chat, document viewing, or other rich interfaces
Results:
- Windows 7: FAIL (Simona)
- Ubuntu 12.04: PASS (Virgil)
- Mac OSX 10.6: PASS (VladG)
Details & Bugs:
- bug 725614 - Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content
- bug 759689 - Flash content is not loaded using Flash 11.3 Beta
- On Windows 7 - Flash content is not loaded using Flash 11.3 beta. Issue is reproducible on Windows 7 with no Service Pack. Flash 11.3 works fine on Windows 7 with SP 1. Flash 11.2 does not cause any problems.

Top Site Compatibility

Tests:
- YouTube: video playback, 3D, uploading large files (>100MB), commenting
- Facebook: chat, status, pictures, video, games, timeline, groups
- Banking website authentication and transactions
- E-Store authentication, shopping cart, and checkout
- keep an eye out for crashes
Results:
- Windows 7: PASS (Mihaela)
- Ubuntu 12.04: PASS (Paul)
- Mac OSX 10.6: PASS (VladG)
Details & Bugs:

Mac OS X 10.8

Tests:
- Try all the menu item functions
- Gestures
- Plug-in compatibility
- Top-site compatibility
- Add-on compatibility
- Software update, especially with rich profiles and fallback updates (use betatest channel)
Results: PASS (VladG)
Details & Bugs:

Windows 8

Tests:
- Try all the menu item functions
- Plug-in compatibility
- Top-site compatibility
- Add-on compatibility
- Software update, especially with rich profiles and fallback updates (use betatest channel)
Results: FAIL (Paul)
Details & Bugs:
- bug 760028 - Tabs change their colors
- bug 760023 - Firefox doesn't behave as the default browser

Exploratory testing

Exploratory testing on add-ons, plugins, top sites and new features Etherpad

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox13 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]

Query	Triaged	Remaining	% Complete
QA+: for testers to verify functional fixes	106	13	89%
QA+ SG: for the Security team to verify security fixes	22	9	71%
status-firefox13:fixed: for QA lead to triage for potential QA+ bugs	379	0	100%

Softvision verifications etherpad

QAWANTED

QAWANTED Bugs

bug 719117 Crash @ mozalloc_abort NS_DebugBreak_P nsIFrame::GetOffsetToCrossDoc
- VERIFIED FIXED
bug 756289 Netflix video errors "you are viewing from more than one tab"
- Marcia, any information you can add to aid debugging?
bug 757262 Youtube keeps playing audio after closing
- IU has an unconfirmed testcase which takes ~1hr to test, can it be reduced?
- Softvision tested but was unable to reproduce the issue on Win/Linux, still need to test Mac

Issues Found

Full Query

ID	Summary	Priority	Status
759689	[adbe 3210913] Flash content is not loaded using Flash 11.3 Beta	--	RESOLVED
760119	Switching back from custom install to standard doesn't reset the installation folder	--	RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);

Known Issues

Full Query

ID	Summary	Priority	Status
585971	autocomplete matches unrelated hosts when searching for www.example.com	--	RESOLVED
667399	RealPlayer G2 LiveConnect plugin crash	--	RESOLVED
692446	Firefox prompts to install windows media player plugin even though it is already installed	--	RESOLVED
725296	Browser is unresponsive while Silverlight content is loading	--	RESOLVED
725614	Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content	P5	RESOLVED
726943	Fullscreen button in HTML video makes the browser go fullscreen but not the video	--	VERIFIED
727702	Undoing applying a Persona reverts back to default theme instead of old Persona	--	VERIFIED
730829	Restore "Open all in tabs" to the bottom of livemarks popups (was: "Open all in tabs" missing after bug 613588)	--	RESOLVED
749112	"Live Bookmarks Loading" message is displayed at the bottom of the RSS feeds	--	RESOLVED
753680	Cannot enable all personas from Add-ons Manager list	--	VERIFIED
753684	Removing an add-on installed twice doesn't actually remove it	--	RESOLVED
753770	Printing progress bar is not properly displayed	--	RESOLVED
753889	stage.mo -> releases.mo breaks staging beta builds for update testing	--	VERIFIED
754829	In Full Screen mode - YouTube videos player changes dimmension	--	RESOLVED
756080	The Bookmarks toolbar is set as visible automatically when resetting Firefox with at least 5 bookmarks toolbar items	--	RESOLVED
756111	Emoticons are not displayed on the Facebook chat	--	RESOLVED

16 Total; 0 Open (0%); 12 Resolved (75%); 4 Verified (25%);

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: FTP, HTTP

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 7


Builds	FTP, HTTP
ID	20120531155942
Changeset	91bf09009594

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[ON TRACK] Bug fix verifications (target 100%)
[ON TRACK] Security fix verifications (target 100%)
[DONE] Update tests on releasetest
[DONE] Get stability sign-off (marcia)
[DONE] Get final sign-off (bmoss)
[DONE] Update tests on beta

Issues of Note

[NOT BLOCKING] bug 760415 Firefox crashes if locking and unlocking Windows XP while on http://chrome.angrybirds.com/

Will fix for Firefox 14, QA is working on a regression range
Only affects those with WebGL turned on (very small userbase)
Intermittent in Firefox 11, prevalent in Firefox 12 (if it were serious, we'd have heard about it already)

Automated Tests

Results of the Mozmill automation testruns.

Functional Tests

Platform	Result: report	Failures: known bugs
Windows XP	PASS - 2012-05-31	0 - no fails
Windows Vista	PASS - 2012-05-31	0 - no fails
Windows 7 32-bit	PASS - 2012-05-31	0 - no fails
Windows 7 64-bit	PASS - 2012-05-31	0 - no fails
Mac OS X 10.6	PASS - 2012-05-31	0 - no fails
Mac OS X 10.7	PASS - 2012-05-31	0 - no fails
Ubuntu 11.10 32-bit	PASS - 2012-05-31	1 - testAboutPrivateBrowsing.js
Ubuntu 11.10 -bit	PASS - 2012-05-31	0 - no fails

Update Tests

Platform	Releasetest: report	Beta: report
Windows XP	PASS - 2012-06-01	PASS - 2012-06-01
Windows Vista	PASS - 2012-06-01	PASS - 2012-06-01
Windows 7 32-bit	PASS - 2012-06-01	PASS - 2012-06-01
Windows 7 64-bit	PASS - 2012-06-01	PASS - 2012-06-01
Mac OS X 10.6	PASS - 2012-06-01	PASS - 2012-06-01
Mac OS X 10.7	PASS - 2012-06-01	PASS - 2012-06-01
Ubuntu 11.10 32-bit	PASS - 2012-06-01	PASS - 2012-06-01
Ubuntu 11.10 -bit	PASS - 2012-06-01	PASS - 2012-06-01

Smoketests

Run the Firefox 13.0 Smoketests in Litmus and report results below.

Platform	Assignee	Result: report
Windows XP	Ioana	FAIL - 2012-06-01
Windows 7 64-bit	Paul	PASS - 2012-06-01
Mac OS X 10.6.8	VladG	PASS - 2012-06-01
Mac OS X 10.7.4	VladG	PASS - 2012-06-01
Ubuntu 11.10 64-bit	Ioana	PASS - 2012-06-01
Ubuntu 12.04 32-bit	Virgil	PASS - 2012-06-01

Focus Testing

Focus Area	Win32	Mac	Linux
Plug-ins	PASS - 2012-06-01	PASS - 2012-06-01	PASS - 2012-06-01
Top-sites	PASS - 2012-06-01	PASS - 2012-06-01	PASS - 2012-06-01
Mac OSX 10.8	N/A	PASS - 2012-06-01	N/A

[RESOLVED] Priority: bug 760190

NOTE - focus here no longer needed, please refocus on usual sign-off

bug 760190 OOM crash in `anonymous namespace'::CSSParserImpl::ParseSelector or ParseSelectorGroup
Try anything you can think of to reproduce the crash -- we have no leads and this is a blocker!

Install Java (6u31 seems to have the highest correlation)
Windows XP and 7
Start with Firefox 13.0b6
Start testing http://scratch.mit.edu/ and then branch out to other heavy Java sites
Once we have STR, test 13.0b7 -- we need to see if it's fixed or not

[UPDATE] Correlated to 32-bit Windows with Java 6 on a netbook with 2GB of RAM (test under similar conditions)

[UPDATE] We now have STR and hardware, no need to test this; just focus on Plugins testing as below

Plug-ins

Tests:
- Flash (11.2, 11.3)
- Java
- Silverlight
- QuickTime
- Try videos, games, chat, document viewing, maps, street view, or other rich interfaces
Results:
- Windows 7: PASS (Simona)
- Ubuntu 12.04: PASS (Virgil)
- Mac OSX 10.6: PASS (VladG)
Details & Bugs:
- bug 759689 - Flash content is not loaded using Flash 11.3 Beta
- bug 675645 - Firefox freezes when using Pop out while in Fullscreen mode on Youtube

Top Site Compatibility

Tests:
- YouTube: video playback, 3D, uploading large files (>100MB), commenting
- Facebook: chat, status, pictures, video, games, timeline, groups
- Banking website authentication and transactions
- E-Store authentication, shopping cart, and checkout
- keep an eye out for crashes
Results:
- Windows 7: PASS (Simona)
- Ubuntu 12.04: PASS (Virgil)
- Mac OSX 10.6: PASS (VladG)
Details & Bugs:

Mac OS X 10.8

Tests:
- Before testing run codesign -vv Firefox.app on the file to ensure it is signed
  - I getting the following message: valid on disk, satisfies its Designates Requirements
  - The build is signed
- Try all the menu item functions, including
- Gestures
- Plug-in compatibility (as above)
- Top-site compatibility (as above)
- Software update, especially with rich profiles and fallback updates (use betatest channel)
Results: PASS (VladG)
Details & Bugs:

Bug Fix Verifications

P1 Verifications (DO THESE FIRST!)

bug 757262 Youtube keeps playing audio after closing
- Can't verify due to unreproducible case, will defer to Beta feedback
bug 759788 Java related Crash (sg:crit)
- Currently testing...

P2 Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox13 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]

Query	Triaged	Remaining	% Complete
QA+: for testers to verify functional fixes	107	12	87%
QA+ SG: for the Security team to verify security fixes	24	7	77%
status-firefox13:fixed: for QA lead to triage for potential QA+ bugs	379	0	100%

Softvision verifications etherpad

QAWANTED

QAWANTED Bugs

bug 756289 Netflix video errors "you are viewing from more than one tab"

Issues Found

Full Query

ID	Summary	Priority	Status
759689	[adbe 3210913] Flash content is not loaded using Flash 11.3 Beta	--	RESOLVED
760119	Switching back from custom install to standard doesn't reset the installation folder	--	RESOLVED
760415	Firefox crashes if locking and unlocking Windows XP while on http://chrome.angrybirds.com/	--	RESOLVED
760417	When selecting "Pop out" for HTML5 videos, the window is opened in full screen mode and the "Full screen" button is still available on its right bottom side	--	RESOLVED

4 Total; 0 Open (0%); 4 Resolved (100%); 0 Verified (0%);

Release Candidate


Builds	FTP, HTTP
ID	20120601045813
Changeset	2b643ea8edf9

Checklist

[DONE] Functional automated tests, synced to Litmus
[DONE] Manual tests
[DONE] Bug fix verifications (target 100%)
- 2 remaining fixes pending Engineering assistance to test
[DONE] Security fix verifications (target 100%)
[DONE] Update tests on releasetest
[DONE] Get stability sign-off (marcia)
[DONE] Get final sign-off (bmoss)
[DONE] Update tests on release

Automated Tests

Results of the Mozmill automation testruns.

Functional Tests

Platform	Result: report	Failures: known bugs
Windows XP	PASS - 2012-06-01	0 - no fails
Windows Vista	PASS - 2012-06-01	0 - no fails
Windows 7 32-bit	PASS - 2012-06-01	0 - no fails
Windows 7 64-bit	PASS - 2012-06-01	0 - no fails
Mac OS X 10.6	PASS - 2012-06-01	0 - no fails
Mac OS X 10.7	PASS - 2012-06-01	0 - no fails
Ubuntu 11.10 32-bit	PASS - 2012-06-01	0 - no fails
Ubuntu 11.10 -bit	PASS - 2012-06-01	0 - no fails

Update Tests

Firefox 10 through 12 -> 13

Platform	Releasetest: report	Release: report
Windows XP	PASS - 2012-06-02	PASS - 2012-06-05
Windows Vista	PASS - 2012-06-02	PASS - 2012-06-05
Windows 7 32-bit	PASS - 2012-06-02	PASS - 2012-06-05
Windows 7 64-bit	PASS - 2012-06-02	PASS - 2012-06-05
Mac OS X 10.6	PASS - 2012-06-02	PASS - 2012-06-05
Mac OS X 10.7	PASS - 2012-06-02	PASS - 2012-06-05
Ubuntu 11.10 32-bit	PASS - 2012-06-02	PASS - 2012-06-05
Ubuntu 11.10 -bit	PASS - 2012-06-02	PASS - 2012-06-05

Firefox 4 through 9 -> 12

Platform	Releasetest: report	Release: report
Windows 2000	PASS - 2012-06-02	PASS - 2012-06-05
Windows XP	PASS - 2012-06-02	PASS - 2012-06-05
Windows Vista	PASS - 2012-06-02	PASS - 2012-06-05
Windows 7 32-bit	PASS - 2012-06-02	PASS - 2012-06-05
Windows 7 64-bit	PASS - 2012-06-02	PASS - 2012-06-05
Mac OS X 10.6	PASS - 2012-06-02	PASS - 2012-06-05
Mac OS X 10.7	PASS - 2012-06-02	PASS - 2012-06-05
Ubuntu 11.10 32-bit	PASS - 2012-06-02	PASS - 2012-06-05
Ubuntu 11.10 -bit	PASS - 2012-06-02	PASS - 2012-06-05

Smoketests

Run the Firefox 13.0 Smoketests in Litmus and report results below.

Platform	Assignee	Result: report
Windows XP	Otilia	PASS - 2012-06-02
Windows Vista	VladG	PASS - 2012-06-02
Windows 7 32-bit	Mihaela	PASS - 2012-06-02
Windows 7 64-bit	Otilia	PASS - 2012-06-02
Mac OS X 10.6.8	VladG	PASS - 2012-06-02
Mac OS X 10.7.4	VladG	PASS - 2012-06-02
Ubuntu 11.10 32-bit	Ioana	PASS - 2012-06-02
Ubuntu 11.10 64-bit	Ioana	PASS - 2012-06-02

Focus Testing

Focus Area	Win32	Mac	Linux
Plug-ins	PASS - 2012-06-02	PASS - 2012-06-02	PASS - 2012-06-02
Top-sites	PASS - 2012-06-02	PASS - 2012-06-02	PASS - 2012-06-02
Mac OSX 10.8	N/A	PASS - 2012-06-02	N/A

Plug-ins

Tests:
- Flash (11.2, 11.3)
- Java
- Silverlight
- QuickTime
- Try videos, games, chat, document viewing, or other rich interfaces
Results:
- Windows 7: PASS (Mihaela)
- Ubuntu 11.10: PASS (Ioana)
- Mac OSX 10.6: PASS (VladG)
Details & Bugs:

Top Site Compatibility

Tests:
- YouTube: video playback, 3D, uploading large files (>100MB), commenting
- Facebook: chat, status, pictures, video, games, timeline, groups
- Banking website authentication and transactions
- E-Store authentication, shopping cart, and checkout
- keep an eye out for crashes
Results:
- Windows 7: PASS (Mihaela)
- Ubuntu 11.10: PASS (Ioana)
- Mac OSX 10.6: PASS (VladG)
Details & Bugs:

Mac OS X 10.8

Tests:
- Try all the menu item functions
- Gestures
- Plug-in compatibility
  - With html5 enabled, in you tube, the videos are flickering green along with some video overlay. Perhaps this is caused by the video card drivers for OS X 10.8. I'll investigate further.
- Top-site compatibility
- Add-on compatibility
- Software update, especially with rich profiles and fallback updates (use betatest channel)
Results: PASS with issues (VladG)
Details & Bugs:

Exploratory testing

Tests:
- Add-ons, SPDY, Smooth scrolling, Software update, Reset Firefox, Windows 8, Plugins, On-demand Tab Restore
Results: (etherpad)
- Windows 7: PASS (Mihaela)
- Ubuntu 11.10: PASS (Ioana)
- Mac OSx 10.6: PASS (VladG)
Details & Bugs:
- bug 760810 - Google images results pages don't scroll smoothly with PgUp/PgDn and Up/Down key arrows

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

Any fix you very should have the status-firefox13 flag set to verified
If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]

Query	Triaged	Remaining	% Complete
QA+: for testers to verify functional fixes	112	2	98%
QA+ SG: for the Security team to verify security fixes	30	0	100%
status-firefox13:fixed: for QA lead to triage for potential QA+ bugs	379	0	100%

Softvision verifications etherpad

QAWANTED

QAWANTED Bugs

Speculative fixes to be verified via post-release feedback

bug 756289 Netflix video errors "you are viewing from more than one tab"
bug 757262 Youtube keeps playing audio after closing

Tracking for 13.0.1

bug 760190 OOM crash in `anonymous namespace'::CSSParserImpl::ParseSelector
- Likely fallout from fixed #1 top-crash (bug 719117)
bug 709860 OOM crash in nsIDNService::Init @ mozilla::Preferences::GetBranch
- Long-standing crash seems to have escalated in Fx13, Marcia is investigating

Add-on Specific

bug 760960 crash in nsNPAPIPluginInstance::GetJSObject with LightShot plugin
- Staging a block for the LightShot add-on, developer has taken it down from AMO

Issues Found

Full Query

ID	Summary	Priority	Status
760810	Google images results pages don't scroll smoothly with PgUp/PgDn and Up/Down key arrows	--	RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);

Issues Found

Known Issues

Full Query

ID	Summary	Priority	Status
585971	autocomplete matches unrelated hosts when searching for www.example.com	--	RESOLVED
587140	Tab groups can push other groups across screen boundaries, making them invisible or causing groups to overlap	P3	RESOLVED
667399	RealPlayer G2 LiveConnect plugin crash	--	RESOLVED
692446	Firefox prompts to install windows media player plugin even though it is already installed	--	RESOLVED
725296	Browser is unresponsive while Silverlight content is loading	--	RESOLVED
725614	Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content	P5	RESOLVED
726943	Fullscreen button in HTML video makes the browser go fullscreen but not the video	--	VERIFIED
727702	Undoing applying a Persona reverts back to default theme instead of old Persona	--	VERIFIED
730829	Restore "Open all in tabs" to the bottom of livemarks popups (was: "Open all in tabs" missing after bug 613588)	--	RESOLVED
749112	"Live Bookmarks Loading" message is displayed at the bottom of the RSS feeds	--	RESOLVED
753680	Cannot enable all personas from Add-ons Manager list	--	VERIFIED
753684	Removing an add-on installed twice doesn't actually remove it	--	RESOLVED
753770	Printing progress bar is not properly displayed	--	RESOLVED
753889	stage.mo -> releases.mo breaks staging beta builds for update testing	--	VERIFIED
754829	In Full Screen mode - YouTube videos player changes dimmension	--	RESOLVED
756080	The Bookmarks toolbar is set as visible automatically when resetting Firefox with at least 5 bookmarks toolbar items	--	RESOLVED
756111	Emoticons are not displayed on the Facebook chat	--	RESOLVED
759689	[adbe 3210913] Flash content is not loaded using Flash 11.3 Beta	--	RESOLVED
760119	Switching back from custom install to standard doesn't reset the installation folder	--	RESOLVED
760415	Firefox crashes if locking and unlocking Windows XP while on http://chrome.angrybirds.com/	--	RESOLVED
760417	When selecting "Pop out" for HTML5 videos, the window is opened in full screen mode and the "Full screen" button is still available on its right bottom side	--	RESOLVED

21 Total; 0 Open (0%); 17 Resolved (80.95%); 4 Verified (19.05%);

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: FTP, HTTP

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

13.0.1


Builds	FTP, HTTP
ID	20120614114901
Changeset	f48d675ffa9f

Automated Tests

Results of the Mozmill automation testruns.

Functional Tests

Platform	Result: report	Failures: known bugs
Windows XP	PASS - 2012-06-14	0 - no fails
Windows Vista	PASS - 2012-06-14	0 - no fails
Windows 7 32-bit	PASS - 2012-06-14	0 - no fails
Windows 7 64-bit	PASS - 2012-06-14	0 - no fails
Mac OS X 10.6	PASS - 2012-06-14	0 - no fails
Mac OS X 10.7	PASS - 2012-06-14	0 - no fails
Ubuntu 11.10 32-bit	PASS - 2012-06-14	0 - no fails
Ubuntu 11.10 -bit	PASS - 2012-06-14	0 - no fails

Update Tests

Firefox 10.0 through 13.0 -> 13.0.1

Platform	Releasetest: report	Release: report
Windows XP	PASS - 2012-06-15	PASS - 2012-06-15
Windows Vista	PASS - 2012-06-15	PASS - 2012-06-15
Windows 7 32-bit	PASS - 2012-06-15	PASS - 2012-06-15
Windows 7 64-bit	PASS - 2012-06-15	PASS - 2012-06-15
Mac OS X 10.6	PASS - 2012-06-15	PASS - 2012-06-15
Mac OS X 10.7	PASS - 2012-06-15	PASS - 2012-06-15
Ubuntu 11.10 32-bit	PASS - 2012-06-15	PASS - 2012-06-15
Ubuntu 11.10 -bit	PASS - 2012-06-15	PASS - 2012-06-15

Firefox 4.0 through 9.0 -> 12.0

RELEASETEST
- Firefox 4.0 win32 en-US: PASS - 2012-06-15
- Firefox 8.0 mac de: PASS - 2012-06-15
- Firefox 9.0 linux pt-BR: FAIL - 2012-06-15
  - bug 765312 - FF4 to FF9 Linux 32/64-bit users 404 for complete.mars
RELEASE
- Firefox 4.0 win32 en-US: PASS - 2012-06-15
- Firefox 8.0 mac de: PASS - 2012-06-15
- Firefox 9.0 linux pt-BR: PASS - 2012-06-15

What's New Page

RELEASETEST
- Firefox 13.0 win32 en-US - PASS - 2012-06-15 (no WN page)
- Firefox 12.0 mac ja-JP-mac - PASS - 2012-06-15
- Firefox 11.0 linux de - PASS - 2012-06-15
RELEASE
- Firefox 13.0 win32 en-US - PASS - 2012-06-15 (no WN page)
- Firefox 12.0 mac ja-JP-mac - PASS - 2012-06-15
- Firefox 11.0 linux de - PASS - 2012-06-15

Manual Testing

Focus Area	Win32	Mac	Linux
Plug-ins	PASS - 2012-06-15	PASS - 2012-06-15	PASS - 2012-06-15
Top-sites	PASS - 2012-06-15	PASS - 2012-06-15	PASS - 2012-06-15
Mac OSX 10.8	N/A	PASS - 2012-06-15	N/A

Details

Plug-ins

Tests:
- Flash 11.3
- Java 7u4
- Silverlight
- QuickTime
- Try videos, games, chat, document viewing, or other rich interfaces, closing tabs/windows with loaded plug-in content
Results:
- Windows 7: PASS (Paul)
- Ubuntu 12.04: PASS (Virgil)
- Mac OSX 10.6.8: PASS (VladG)
Details & Bugs:

Top Site Compatibility

Tests:
- YouTube: video playback, 3D, uploading large files (>100MB), commenting
- Facebook: chat, status, pictures, video, games, timeline, groups
- Twitter: tweet, follow, unfollow, media sharing, trends
- Banking website authentication and transactions
- E-Store authentication, shopping cart, and checkout
- Webmail: Hotmail, Gmail, Yahoo
- keep an eye out for crashes
Results:
- Windows 7: PASS (Paul)
- Ubuntu 12.04: PASS (Virgil)
- Mac OSX 10.6.8: PASS (VladG)
Details & Bugs:

Mac OS X 10.8 Code-signing

NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (more info).

Download the .dmg file and drag the Firefox.app file to a location on your computer
- PASS -VladG
Open a terminal window and cd to the folder Firefox.app is located
- PASS -VladG
Run codesign -vv Firefox.app in terminal, output should contain valid on disk and satisfies its Designated Requirement
- PASS -VladG
Run xattr Firefox.app in terminal, output should contain com.apple.quarantine
- PASS -VladG
Double click the Firefox.app icon, you should see a warning about the program being provided by an unknown developer or having been downloaded from the Internet
- PASS -VladG
Click "open" and browse to any website, Firefox should start and you should be able to browse
- PASS -VladG
Quit Firefox and double click the Firefox.app icon again, you should not be reprompted for the warning in step 5, firefox should start up and browse without any issues
- PASS -VladG

Do the following testing to make sure the build works as expected...

Tests:
- Try all the menu item functions
- Gestures
- Plug-in compatibility
  - With html5 enabled, in you tube, the videos are flickering green along with some video overlay. Perhaps this is caused by the video card drivers for OS X 10.8. I'll investigate further.
  - I've got no flickering and the videos are playing as expected in html5. The test machine has a AMD Radeon HD 6630M 256 MB video card - VladG
- Top-site compatibility
- Add-on compatibility
- Software update, especially with rich profiles and fallback updates (use betatest channel)
Results: PASS with issues (VladG)
Details & Bugs:
- bug 736655 - (10.8) Mac dmg installer dialog doesn't look right

Bug Verifications

The following should be solved via Blocklisting:

[ON TRACK] bug 763459 - RealPlayer Browser Record plugin breaks Youtube since Fx13
[ON TRACK] bug 762062 - Firefox startup crash in JS_HoldPrincipals (Skype 6.0.0.10201)

Fixed bugs needing Verification:

[DROPPED] bug 756277 - XMLHttpRequest started from Cc["@mozilla.org/xmlextras/xmlhttprequest;1"] ends up bound to some random window
[DONE] bug 756850 - Some combinations of Hebrew vowels and consonants not displayed in certain fonts with harfbuzz
[DONE] bug 747683 [adbe 3172259] Crash Report [@ LdrShutdownProcess ] with Flash 11.3 "Protected Mode for Firefox"

Backout bugs needing Confirmation:

[DONE] bug 733614 - CSS 'height' property on a multi-column block is not respected
[DONE] bug 736731, bug 764546 - Hotmail no longer auto-updates (due to "globalStorage" removal)

Third-party bugs we are tracking but won't resolve on our end:

bug 681521 - Firefox Crash [@ JS_GetContextPrivate ] (Trusteer)
bug 761598 - Crash in RapportTanzan13 with Trusteer Rapport and LastPass
bug 709860 - OOM crash in nsIDNService::Init @ mozilla::Preferences::GetBranch (Avast)
bug 759689 - [adbe 3210913] Flash content is not loaded using Flash 11.3 Beta

13.0.2


Builds	[ FTP], [ HTTP]
ID
Changeset	[]

Automated Tests

Results of the Mozmill automation testruns.

Functional Tests

Platform	Result: [ report]	Failures: known bugs
Windows XP	result - YYYY-MM-DD	0 - no fails
Windows Vista	result - YYYY-MM-DD	0 - no fails
Windows 7 32-bit	result - YYYY-MM-DD	0 - no fails
Windows 7 64-bit	result - YYYY-MM-DD	0 - no fails
Mac OS X 10.6	result - YYYY-MM-DD	0 - no fails
Mac OS X 10.7	result - YYYY-MM-DD	0 - no fails
Ubuntu 11.10 32-bit	result - YYYY-MM-DD	0 - no fails
Ubuntu 11.10 -bit	result - YYYY-MM-DD	0 - no fails

Update Tests

Firefox 10.0 through 13.0 -> 13.0.1

Platform	Releasetest: [ report]	Release: [ report]
Windows XP	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows Vista	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 7 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Windows 7 64-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS X 10.6	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OS X 10.7	result - YYYY-MM-DD	result - YYYY-MM-DD
Ubuntu 11.10 32-bit	result - YYYY-MM-DD	result - YYYY-MM-DD
Ubuntu 11.10 -bit	result - YYYY-MM-DD	result - YYYY-MM-DD

Firefox 4.0 through 9.0 -> 12.0

RELEASETEST
- Firefox 4.0 win32 en-US: result - YYYY-MM-DD
- Firefox 8.0 mac de: result - YYYY-MM-DD
- Firefox 9.0 linux pt-BR: result - YYYY-MM-DD
RELEASE
- Firefox 4.0 win32 en-US: result - YYYY-MM-DD
- Firefox 8.0 mac de: result - YYYY-MM-DD
- Firefox 9.0 linux pt-BR: result - YYYY-MM-DD
Automatic Spotcheck

What's New Page

RELEASETEST
- Firefox 13.0 win32 en-US - result - YYYY-MM-DD (no WN page)
- Firefox 12.0 mac ja-JP-mac - result - YYYY-MM-DD
- Firefox 11.0 linux de - result - YYYY-MM-DD
RELEASE
- Firefox 13.0 win32 en-US - result - YYYY-MM-DD (no WN page)
- Firefox 12.0 mac ja-JP-mac - result - YYYY-MM-DD
- Firefox 11.0 linux de - result - YYYY-MM-DD

Manual Testing

Focus Area	Win32	Mac	Linux
Plug-ins	result - YYYY-MM-DD	result - YYYY-MM-DD	result - YYYY-MM-DD
Top-sites	result - YYYY-MM-DD	result - YYYY-MM-DD	result - YYYY-MM-DD
Mac OSX 10.8	N/A	result - YYYY-MM-DD	N/A

Plug-ins

Tests:
- Flash 11.3
- Java 7u4
- Silverlight
- QuickTime
- Try videos, games, chat, document viewing, or other rich interfaces, closing tabs/windows with loaded plug-in content
Results:
- Windows 7: result - YYYY-MM-DD (tester)
- Ubuntu 12.04: result - YYYY-MM-DD (tester)
- Mac OSX 10.6.8: result - YYYY-MM-DD (tester)
Details & Bugs:

Top Site Compatibility

Tests:
- YouTube: video playback, 3D, uploading large files (>100MB), commenting
- Facebook: chat, status, pictures, video, games, timeline, groups
- Twitter: tweet, follow, unfollow, media sharing, trends
- Banking website authentication and transactions
- E-Store authentication, shopping cart, and checkout
- Webmail: Hotmail, Gmail, Yahoo
- keep an eye out for crashes
Results:
- Windows 7: result - YYYY-MM-DD (tester)
- Ubuntu 12.04: result - YYYY-MM-DD (tester)
- Mac OSX 10.6.8: result - YYYY-MM-DD (tester)
Details & Bugs:

Mac OS X 10.8 Code-signing

NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test (more info).

Download the .dmg file and drag the Firefox.app file to a location on your computer
- result - YYYY-MM-DD (tester)
Open a terminal window and cd to the folder Firefox.app is located
- result - YYYY-MM-DD (tester)
Run codesign -vv Firefox.app in terminal, output should contain valid on disk and satisfies its Designated Requirement
- result - YYYY-MM-DD (tester)
Run xattr Firefox.app in terminal, output should contain com.apple.quarantine
- result - YYYY-MM-DD (tester)
Double click the Firefox.app icon, you should see a warning about the program being provided by an unknown developer or having been downloaded from the Internet
- result - YYYY-MM-DD (tester)
Click "open" and browse to any website, Firefox should start and you should be able to browse
- result - YYYY-MM-DD (tester)
Quit Firefox and double click the Firefox.app icon again, you should not be reprompted for the warning in step 5, firefox should start up and browse without any issues
- result - YYYY-MM-DD (tester)

Do the following testing to make sure the build works as expected...

Tests:
- Try all the menu item functions
- Gestures
- Plug-in compatibility
  - With html5 enabled, in you tube, the videos are flickering green along with some video overlay. Perhaps this is caused by the video card drivers for OS X 10.8. I'll investigate further.
  - I've got no flickering and the videos are playing as expected in html5. The test machine has a AMD Radeon HD 6630M 256 MB video card - VladG
- Top-site compatibility
- Add-on compatibility
- Software update, especially with rich profiles and fallback updates (use betatest channel)
Results: result - YYYY-MM-DD (tester)
Details & Bugs:

Bug Verifications

Add your name beside whatever you are testing so people don't duplicate your results.

bug 686335 - Bad plugin search order prevents upgrading Flash without an OS restart
bug 758363 - Implement NPN_ReloadPlugins for out-of-process Plugins

NOTE that testing any of these in a VM requires the VM to be run in Protected Mode

My advice would be to take a snapshot of the VM before starting and rolling back before each test they can be run from a clean snapshot.

NOTE that not all of these plugin versions are available for Mac and Linux.

Just test with whatever old versions you can find and update to the latest. Report the version numbers in your results.

Builds

Firefox 16.0a1 Nightly: FTP
Patched 13.0.1 Tinderbox: win32, linux64, linux, mac
13.0.2 Candidates:

Tests

Flash 10.2 (blocklisted)

Download and install Flash 10.2
Blocklist Flash by forcing a ping
Load some Flash content in tabs before updating
Pave-over install Flash 11.3.300.262
Reload tabs with Flash

Flash 10.3 (enabled & disabled)

Download and install Flash 10.3
Load some Flash content in tabs before updating
Pave-over install Flash 11.3.300.262
Reload tabs with Flash
Uninstall Flash and retest with Flash disabled before updating

Flash 11.2 (enabled & disabled)

Download and install Flash 11.2
Load some Flash content in tabs before updating
Pave-over install Flash 11.3.300.262
Reload tabs with Flash
Uninstall Flash and retest with Flash disabled before updating

Flash 11.3 (enabled & disabled)

Download and install Flash 11.3.300.257
Load some Flash content in tabs before updating
Pave-over Flash 11.3.300.262
Uninstall Flash and retest with Flash disabled before updating

Flash 11.3 Automatic Update (dom.ipc.plugins.enabled=true)

Download and install Flash 11.2
Start Firefox and open a page with flash content
Download but do not install Flash 11.3
Open a command line to the download folder and run:
- install_flash_player.exe -install -iv 9
Reload the page with Flash content once the installer finishes
Uninstall Flash
Repeat the test, loading Flash content in a new tab after update
Repeat the test, loading Flash content in a new window after update

Flash 11.3 Automatic Update (dom.ipc.plugins.enabled=false)

Download and install Flash 11.2
Start Firefox and open a page with flash content
Download but do not install Flash 11.3
Open a command line to the download folder and run:
- install_flash_player.exe -install -iv 9
Reload the page with Flash content once the installer finishes
Uninstall Flash
Repeat the test, loading Flash content in a new tab after update
Repeat the test, loading Flash content in a new window after update

Java 6u31 (enabled & disabled)

Download and install Java 6u31
Load some Java content in tabs before updating
Pave-over install Java 7u5
Reload tabs with Java content after updating
Uninstall Java and retest with Java disabled before updating

Java 7u2 (enabled & disabled)

Download and install Java 7u2
Load some Java content in tabs before updating
Pave-over install Java 7u5
Reload tabs with Java content after updating
Uninstall Java and retest with Java disabled before updating

Results

Expected

Plugin updates without any problems
Latest version of the plugin is listed in about:plugins
Content loads (or reloads) after update without crashing
NOTE: Multiple versions may be listed in about:plugins
- OKAY as long as plugin content is using the latest version
- Multiple entries can be cleared by restarting the OS

Windows 7

Firefox 16.0a1 2012-06-22 (ashughes)
- PASS Flash 10.2 (blocklisted)
- PASS Flash 10.3 (enabled & disabled)
- PASS Flash 11.2 (enabled & disabled)
- ISSUE Flash 11.3 (enabled & disabled)
  - Multiple entries, crash when not using auto-update
- ISSUE Java 6u31 (enabled & disabled)
  - Firefox requires restart before updated plug-in is activated
- PASS Java 7u2 (enabled & disabled)
- PASS Flash 11.3 Automatic Update (dom.ipc.plugins.enabled=true)
- ISSUE Flash 11.3 Automatic Update (dom.ipc.plugins.enabled=false)
  - dom.ipc.plugins.enabled=false disables Flash (unreproducible)

Windows XP

Firefox 16.0a1 2012-06-22 (ashughes)
- PASS Update Flash 10.2 -> 11.3.300.262 (enabled) using the installer method
- PASS Update Flash 11.2 -> 11.3.300.262 (enabled) using the auto-update method
- ISSUE Update Java 6u31 -> 7u5 (disabled)
  - Firefox requires restart before updated plug-in is activated
- PASS Update Java 7u2 -> 7u5 (enabled)

Mac OSX 10.6

Firefox 16.0a1 2012-06-22 (ashughes)
- PASS Update Flash 10.2 -> 11.3.300.257 (enabled) via pave-over install
- PASS Update Flash 11.2 -> 11.3.300.257 (enabled) via pave-over install
- Note: Max Flash version is 11.3.300.257
- Could not find a way to do Java updates reliably -- could not find an intuitive way to install out of date plugins and upgrade

Ubuntu 12.04

Firefox 13.0.1 Tryserver (virgil)
- PASS Flash 10.2 r159 -> 11.2.205.236 (enabled & disabled) via pave-over
- PASS Flash 11.2.205.235 -> 11.2.205.236 (enabled & disabled) via pave-over
- PASS Flash 10.2 r159 -> 11.2.205.236 (dom.ipc.plugins.enabled=false) via pave-over
- PASS Flash 11.2.205.235 -> 11.2.205.236 (dom.ipc.plugins.enabled=false) via pave-over
Notes
- Flash Plugin: Replace flash by overwriting usr/lib/mozilla/plugins/libflashplayer.so
- IcedTea Java Plugin: was unable to get an older version working in the OS, Software Centre only provides the latest.
- Oracle Java: Could not get older versions to install using this method

Issues Found

Disabled Java requires restart of Firefox to re-enable after update
Side-by-Side Flash versions in about:config when paving over Flash 11.3.300.257 64-bit with 11.3.300.262 32-bit; resolved only by OS restart; does not seem to affect functionality/stability
Side-by-Side Flash versions in about:config when simulating 11.2->11.3 auto-update with install_flash_player.exe -install -iv 9
dom.ipc.plugins.enabled set to false appears to disable Flash
Hang/crash when watching Youtube on update (bug 686335#c63) -- not reproducible using auto-update

12.0 to 13.0.1 Partial Updates

RE bug 768576

Platform	Betatest: report	Releasetest: report	Release: report
Windows XP	PASS - 2012-06-27	PASS - 2012-06-28	PASS - 2012-06-28
Windows Vista	PASS - 2012-06-27	PASS - 2012-06-28	PASS - 2012-06-28
Windows 7 32-bit	PASS - 2012-06-27	PASS - 2012-06-28	PASS - 2012-06-28
Windows 7 64-bit	PASS - 2012-06-27	PASS - 2012-06-28	PASS - 2012-06-28
Mac OS X 10.6	PASS - 2012-06-27	PASS - 2012-06-28	PASS - 2012-06-28
Mac OS X 10.7	PASS - 2012-06-27	PASS - 2012-06-28	PASS - 2012-06-28
Ubuntu 11.10 32-bit	n/a	n/a	n/a
Ubuntu 11.10 -bit	n/a	n/a	n/a

Issues

[FIXED] bug 769156 - 12->13.0.1 updates on releasetest fail due to 13.0.2build1 staging
- Resolved by bhearsum deleting 13.0.1->13.0.2 snippets on releasetest channel
- NOTE: All releasetest failures were caused by this bug, cannot be removed from the report

Releases/Firefox 13/Test Plan

Contents

Firefox 13 Test Plan

Summary

Details

Nightly

Merge

Features

Endurance Automation

Functional Automation

Unconfirmed Bug Triage

Aurora Merge

Updates

Manual Spot Checks

Aurora Feature Sign-off

QAWANTED

Beta 1

Automated Tests

Manual Tests

Smoketests

Regression Testing

Feature Testing

Update Tests

Bug Fix Verifications

QAWANTED

Issues Found

Known Issues

Bugzilla query error

Feedback

Beta 2

Automated Tests

Manual Tests

Smoketests

Regression Testing

Feature Testing

Update Tests

Bug Fix Verifications

QAWANTED

Issues Found

Known Issues

Feedback

Beta 3

Automated Tests

Manual Tests

Smoketests

Regression Testing

Feature Testing

Update Tests

Bug Fix Verifications

QAWANTED

Issues Found

Known Issues

Feedback

Beta 4

Automated Tests

Manual Tests

Smoketests

Regression Testing

Feature Testing

Update Tests

Bug Fix Verifications

QAWANTED

Issues Found

Known Issues

Feedback

Beta 5

Automated Tests

Manual Tests

Smoketests

Regression Testing

Feature Testing

Update Tests

Bug Fix Verifications

QAWANTED

Issues Found

Known Issues

Feedback

Beta 6

Automated Tests

Functional Tests