Releases/Firefox 12/Test Plan

From MozillaWiki
Jump to: navigation, search

« Firefox 11 | Firefox 12 | Firefox 13 »

Firefox 12 Test Plan

Summary

Milestone Firefox 12
Status Currently in Beta -- Moves to Release on April 24, 2012
Lead Juan Becerra (irc: juanb)
2nd Anthony Hughes (irc: ashughes)

The following is the test plan for Firefox 12 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 12. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

Task Owner Status
Automation Triage Softvision
Bug Triage Softvision
Feature Sign-offs Softvision
Full Functional Tests (Aurora) iBeta
Website Compatibility (Aurora) uTest
Milestones
Aurora Merge ashughes
Nightly Merge ashughes

Automation

BFTs

Task Triage the Mozmill dashboard for test failures and regressions
Lead Vlad Maniac (Softvision)
Peer Alex Lakatos (Softvision)
Status NEW (13), ASSIGNED (15), FIXED (3), REGRESSIONS (0)
Instructions
  • Check the Mozmill dashboard for failing tests for the day
  • Cross reference any failure to the list of NEW and ASSIGNED bugs
    • If the bug is NEW, assign it to yourself and try to fix it
    • If the bug is ASSIGNED and there has been no activity on the bug in over a week, offer your assistance to fix the bug
    • If there is NO bug on file, file it using this template
  • If you need assistance, ask ashughes in the #mozmill channel
Bugs Found

Endurance

Task Triage the Mozmill Endurance dashboard for test failures and performance regressions
Lead Mihaela Velimiroviciu (Softvision)
Peer
Status Spreadsheet (Nightly)
Instructions
  • Check the dashboard for the performance numbers from the most recent nightly
  • Add the data to the spreadsheet
  • Report any performance spikes or test failures to Dave Hunt and Anthony Hughes
Bugs Found

Bug Triage

Lead Vlad Ghetiu (Softvision)
Peers Virgil Dicu, Ioana Budnar, Mihaela Velimiroviciu, Simona Badau, Paul Silaghi (Softvision)
Queries QA+, FIXED, UNCONFIRMED
Bugs Found

Features

See the Feature Release Tracking wiki for a general overview

[DONE]
[ON TRACK]
[AT RISK]
[MISSED]

Full Functional Tests

The following tracks testing done by iBeta for full functional test coverage. This task is performed once during Aurora only.

  • Lead: Juan Becerra
  • [ link to test suite]
  • [ link to results of testing]
  • [ link to bugs filed by iBeta]

Website Compatibility

The following tracks testing done by uTest for website compatibility. This task is performed after every merge.

  • Lead: Juan Becerra
  • Link to test suite
  • Link to report for Nightly Web Compatibility
  • Link to report for Aurora Web Compatibility
  • Link to report for Beta Web Compatibility
  • Link to bugs filed

Nightly Merge

Verify that the builds generated from the merge to Nightly have updated properly.

Nightly
  • [ON TRACK] [ashughes] Windows XP:
  • [ON TRACK] [ashughes] Windows 7:
  • [ON TRACK] [ashughes] Mac OS X 10.6:
  • [ON TRACK] [ashughes] Ubuntu 11.04:
Issues Found

Aurora Merge

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-02-02
Windows XP PASS - 2012-02-02
Windows Vista PASS - 2012-02-02
Windows 7 32-bit PASS - 2012-02-02
Windows 7 64-bit PASS - 2012-02-02
Mac OS X 10.6 PASS - 2012-02-02
Mac OS X 10.7 PASS - 2012-02-02
Ubuntu 11.04 32-bit PASS - 2012-02-02
Ubuntu 11.04 64-bit PASS - 2012-02-02
Issues Found

Updates

The following are details of update verifications for Firefox 12 merge to Aurora

Auroratest
  • [ON TRACK] [ashughes] Windows XP:
  • [ON TRACK] [ashughes] Windows 7:
  • [ON TRACK] [ashughes] Mac OS X 10.6:
  • [ON TRACK] [ashughes] Ubuntu 11.04:
Aurora
  • [ON TRACK] [ashughes] Windows XP:
  • [ON TRACK] [ashughes] Windows 7:
  • [ON TRACK] [ashughes] Mac OS X 10.6:
  • [ON TRACK] [ashughes] Ubuntu 11.04:
Issues Found

Manual Spot Checks and Feature Sign-Off

Are all features implemented at this point working?

Manual spot Checks [spreadsheet]

  • Platforms: Windows XP, Windows 7 x64, Ubuntu 11.10, Mac 10.6
  • Found known issues:
  • New issues:
  • Status:

Beta 1

Builds: HTTPS
Litmus: testrun, report

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [ON TRACK] Bug fix verifications (10%)
  • [ON TRACK] Check with Support for top issues from previous Beta/Release
  • [DONE] Check with Marcia for top-crashers spiking
  • [ON TRACK] Update tests on releasetest
  • [ON TRACK] Get final sign-off from Bob Moss
  • [ON TRACK] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-03-14
Windows XP ashughes PASS - 2012-03-14
Windows Vista ashughes PASS - 2012-03-14
Windows 7 32-bit ashughes PASS - 2012-03-14
Windows 7 64-bit ashughes PASS - 2012-03-14
Mac OS X 10.6 ashughes PASS - 2012-03-14
Mac OS X 10.7 ashughes PASS - 2012-03-14
Ubuntu 11.10 32-bit ashughes PASS - 2012-03-14
Ubuntu 11.10 64-bit ashughes PASS - 2012-03-14
Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-03-15
Windows XP Ioana PASS - 2012-03-15
Windows Vista Simona PASS - 2012-03-15
Windows 7 32-bit Mihaela PASS - 2012-03-15
Windows 7 64-bit Paul PASS - 2012-03-15
Mac OS X 10.6.8 VladG PASS - 2012-03-15
Mac OS X 10.7.2 VladG PASS - 2012-03-15
Ubuntu 11.10 32-bit Ioana PASS - 2012-03-15
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-15
Focus Testing

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows 7 32-bit Ioana, Mihaela PASS - 2012-03-16
Mac OS X 10.6.8 VladG PASS - 2012-03-15
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-15
Windows XP Simona, Paul PASS - 2012-03-15
  1. [DONE] Rich Profile Updates Etherpad
Feature Sign-off Testing

The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.

Feature Owner Win32 Mac Linux
Media Controls Refresh Mihaela Velimiroviciu PASS - 2012-03-15 PASS - 2012-03-15 PASS - 2012-03-15
Silent Update: Removal of OS security dialog for Windows Simona Badau PASS WITH KNOWN ISSUES - 2012-03-16 bug 735177 NA - 2012-03-15 NA - 2012-03-15

Update Tests

The following are details of update verifications

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-03-15 PASS - 2012-03-16
Windows XP PASS - 2012-03-15 PASS - 2012-03-16
Windows Vista PASS - 2012-03-15 PASS - 2012-03-16
Windows 7 32-bit PASS - 2012-03-15 PASS - 2012-03-16
Windos 7 64-bit PASS - 2012-03-15 PASS - 2012-03-16
Mac OS X 10.6 PASS - 2012-03-15 PASS - 2012-03-16
Mac OS 10.7 PASS - 2012-03-15 PASS - 2012-03-16
Ubuntu 11.10 32-bit PASS - 2012-03-15 PASS - 2012-03-16
Ubuntu 11.10 64-bit PASS - 2012-03-15 PASS - 2012-03-16
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox12 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Queries

Issues Found

Bug Status
bug 736021 - The "list all tabs" button is missing in Firefox 12 beta 1 NEW
bug 734837 - Broken history when navigating to and from cnn.com NEW
bug 735243 - On HTML5 videos Esc key exits from both full screen and Highlighter NEW
bug 725614 - Shortcut keys fail for pages containing Java/Shockwave/PDF NEW
bug 729872 - Window deactivate while in HTML5 fullscreen causes text fields to not show cursor or selection Fixed on FF13

Beta 2

Builds: HTTPS
Litmus: testrun, report

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications (27%)
  • [DONE] Check with Support for top issues from previous Beta/Release
  • [DONE] Check with Marcia for top-crashers spiking
  • [DONE] Update tests on releasetest
  • [ON TRACK] Get final sign-off from Bob Moss
  • [ON TRACK] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 juanb PASS - 2012-03-20
Windows XP juanb PASS - 2012-03-20
Windows Vista juanb PASS - 2012-03-20
Windows 7 32-bit juanb PASS - 2012-03-20
Windows 7 64-bit juanb PASS - 2012-03-20
Mac OS X 10.6 juanb PASS - 2012-03-20
Mac OS X 10.7 juanb PASS - 2012-03-20
Ubuntu 11.10 32-bit juanb PASS - 2012-03-20
Ubuntu 11.10 64-bit juanb PASS - 2012-03-20
Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-03-21
Windows XP Paul PASS - 2012-03-22
Windows Vista Simona PASS - 2012-03-21
Windows 7 32-bit Mihaela PASS - 2012-03-22
Windows 7 64-bit Paul PASS - 2012-03-21
Mac OS X 10.6.8 VladG PASS - 2012-03-22
Mac OS X 10.7.3 VladG PASS - 2012-03-22
Ubuntu 11.10 32-bit Ioana PASS - 2012-03-21
Ubuntu 11.10 64-bit Ioana PASS - 2012-03-21
Focus Testing

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows 7 32-bit Mihaela PASS - 2012-03-22
Mac OS X 10.6.8 VladG PASS - 2012-03-22
Ubuntu 11.10 32-bit Ioana PASS - 2012-03-21
Windows XP Paul PASS - 2012-03-22
  1. [DONE] Rich Profile Updates Etherpad
Feature Sign-off Testing

The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.

Feature Owner Win32 Mac Linux
Media Controls Refresh Mihaela Velimiroviciu PASS - 2012-03-22 PASS - 2012-03-22 PASS - 2012-03-22
Silent Update: Removal of OS security dialog for Windows Simona Badau PASS - 2012-03-22 N/A - 2012-03-22 N/A - 2012-03-22

Update Tests

The following are details of update verifications

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-03-20 PASS - 2012-03-22
Windows XP PASS - 2012-03-20 PASS - 2012-03-22
Windows Vista PASS - 2012-03-20 PASS - 2012-03-22
Windows 7 32-bit PASS - 2012-03-20 PASS - 2012-03-22
Windos 7 64-bit PASS - 2012-03-20 PASS - 2012-03-22
Mac OS X 10.6 PASS - 2012-03-20 PASS - 2012-03-22
Mac OS 10.7 PASS - 2012-03-20 PASS - 2012-03-22
Ubuntu 11.10 32-bit PASS - 2012-03-20 PASS - 2012-03-22
Ubuntu 11.10 64-bit PASS - 2012-03-20 PASS - 2012-03-22
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox12 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Queries

QAWANTED

Bugs: query

Bug Status
bug 714320 Firefox Crash @ nsStyleContext::AddChild with AMD Radeon HD 6xxx series Engineering investigating (2012-03-26)
bug 738661 crash in nmsvc from CovenantEyes Marcia investigating (2012-03-26)
bug 738392 Plugin doesn't render inside a CSS transform on Mac Not reproducible (2012-03-26)
bug 725503 Crash in radhslib.dll from Naomi Internet Filter What more can QA do? (2012-03-26)
bug 722538 block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D What more can QA do? (2012-03-26)
bug 720987 Focus rings keep growing when repeatedly tabbing through elements Unclear what's needed (2012-03-26)

Issues Found

Bug Status
bug 736021 - The "list all tabs" button is missing in Firefox 12 beta NEW
bug 734837 - Broken history when navigating to and from cnn.com NEW
bug 735243 - On HTML5 videos Esc key exits from both full screen and Highlighter NEW
bug 725614 - Shortcut keys fail for pages containing Java/Shockwave/PDF NEW
bug 729872 - Window deactivate while in HTML5 fullscreen causes text fields to not show cursor or selection Fixed on FF13
bug 725296 - Browser is unresponsive while Silverlight content is loading NEW
bug 731981 - Missing steps for missing Shockwave plugin notification NEW
bug 709029 - The RealPlayer plugin crashes when trying to play a RealPlayer video NEW

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 3

Builds: HTTPS
Litmus: testrun, report

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications
  • [DONE] Check with Marcia for top-crashers spiking
  • [DONE] Update tests on releasetest
  • [DONE] Get final sign-off from Bob Moss
  • [DONE] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 jsmith PASS - 2012-03-28
Windows XP jsmith PASS - 2012-03-28
Windows Vista jsmith PASS - 2012-03-28
Windows 7 32-bit jsmith PASS - 2012-03-28
Windows 7 64-bit jsmith PASS - 2012-03-28
Mac OS X 10.6 jsmith PASS - 2012-03-28
Mac OS X 10.7 jsmith PASS - 2012-03-28
Ubuntu 11.10 32-bit jsmith PASS - 2012-03-28
Ubuntu 11.10 64-bit jsmith PASS - 2012-03-28
Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-03-29
Windows XP Ioana PASS - 2012-03-29
Windows Vista Simona PASS - 2012-03-29
Windows 7 32-bit Paul PASS - 2012-03-29
Windows 7 64-bit Paul PASS - 2012-03-29
Mac OS X 10.6.8 VladG PASS - 2012-03-29
Mac OS X 10.7.3 VladG PASS - 2012-03-29
Ubuntu 11.10 32-bit Virgil PASS - 2012-03-29
Ubuntu 11.10 64-bit Ioana PASS - 2012-03-29
Focus Testing

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows 7 32-bit Simona PASS - 2012-03-29
Mac OS X 10.6.8 VladG PASS - 2012-03-29
Ubuntu 11.10 32-bit Virgil PASS - 2012-03-29
Windows XP Ioana PASS - 2012-03-29
  1. [DONE] Rich Profile Updates Etherpad
Feature Sign-off Testing

The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.

Feature Owner Win32 Mac Linux
Media Controls Refresh Mihaela Velimiroviciu PASS - 2012-03-29 PASS - 2012-03-30 PASS - 2012-03-30
Silent Update: Removal of OS security dialog for Windows Simona Badau PASS - 2012-03-30 NA - 2012-03-29 NA - 2012-03-29

Update Tests

The following are details of update verifications

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-03-29 PASS - 2012-03-30
Windows XP PASS - 2012-03-29 PASS - 2012-03-30
Windows Vista PASS - 2012-03-29 PASS - 2012-03-30
Windows 7 32-bit PASS - 2012-03-29 PASS - 2012-03-30
Windos 7 64-bit PASS - 2012-03-29 PASS - 2012-03-30
Mac OS X 10.6 PASS - 2012-03-29 PASS - 2012-03-30
Mac OS 10.7 PASS - 2012-03-29 PASS - 2012-03-30
Ubuntu 11.10 32-bit PASS - 2012-03-29 PASS - 2012-03-30
Ubuntu 11.10 64-bit PASS - 2012-03-29 PASS - 2012-03-30
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox12 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Queries

QAWANTED

Bugs: query

Need Follow-up
Full Query
ID Summary Priority Status
714320 Firefox Crash @ nsStyleContext::AddChild with AMD Radeon HD 6xxx series -- RESOLVED
720987 Focus rings keep growing when repeatedly tabbing through elements -- RESOLVED
738392 Plugin doesn't render inside a CSS transform on Mac -- VERIFIED

3 Total; 0 Open (0%); 2 Resolved (66.67%); 1 Verified (33.33%);


QA Investigating
Full Query
ID Summary Priority Status
722538 block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D -- RESOLVED
738661 crash in nmsvc from CovenantEyes P3 RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);


Issues Found

Full Query
ID Summary Priority Status
709029 The RealPlayer plugin crashes when trying to play a RealPlayer video -- RESOLVED
725296 Browser is unresponsive while Silverlight content is loading -- RESOLVED
725614 Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content P5 RESOLVED
729872 Window deactivate while in HTML5 fullscreen causes text fields to not show cursor or selection -- VERIFIED
731981 Missing steps for missing Shockwave plugin notification -- RESOLVED
734837 SHistory gets broken when navigating to and from cnn.com -- RESOLVED
735243 On HTML5 videos Esc key exits from both full screen and Highlighter -- RESOLVED
740304 "Tell Mozilla about the crash" checkbox is unchecked by default on Linux P2 RESOLVED
740321 Address bar unresponsive when previously focusing a pdf opened with Acrobat -- RESOLVED
740330 Yandex Toolbar crashes Firefox -- RESOLVED

10 Total; 0 Open (0%); 9 Resolved (90%); 1 Verified (10%);


Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 4

Builds: HTTPS
Litmus: testrun, report

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [ON TRACK] Bug fix verifications
  • [DONE] Check with Marcia for top-crashers spiking
  • [DONE] Update tests on releasetest
  • [DONE] Get final sign-off from Bob Moss
  • {{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 jsmith PASS - 2012-04-04
Windows XP jsmith PASS - 2012-04-04
Windows Vista jsmith PASS - 2012-04-04
Windows 7 32-bit jsmith PASS - 2012-04-04
Windows 7 64-bit jsmith PASS - 2012-04-04
Mac OS X 10.6 jsmith PASS - 2012-04-04
Mac OS X 10.7 jsmith PASS - 2012-04-04
Ubuntu 11.10 32-bit jsmith PASS - 2012-04-04
Ubuntu 11.10 64-bit jsmith PASS - 2012-04-04
Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-04-04
Windows XP Ioana PASS - 2012-04-04
Windows Vista Simona PASS - 2012-04-04
Windows 7 32-bit Mihaela PASS - 2012-04-04
Windows 7 64-bit Paul PASS - 2012-04-04
Mac OS X 10.6.8 VladG PASS - 2012-04-04
Mac OS X 10.7.3 VladG PASS - 2012-04-04
Ubuntu 11.04 32-bit Mihaela PASS - 2012-04-04
Ubuntu 11.10 64-bit Ioana PASS - 2012-04-04
Regression Testing

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows 7 32-bit Paul PASS - 2012-04-04
Mac OS X 10.6.8 Vlad PASS - 2012-04-04
Ubuntu 11.10 32-bit Virgil PASS - 2012-04-04
Windows XP Ioana PASS - 2012-04-04
Focus Testing

Test the following across as many platforms as possible (at least one Windows, one Mac, and one Linux)

  • [DONE] Bookmarks - verify full functional bookmarks functionality before and after an update (things like import, backup, restore, rearranging, adding, deleting, folders, feeds, toolbars, etc) - Etherpad
Feature Sign-off Testing

The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.

Feature Owner Win32 Mac Linux
Media Controls Refresh Mihaela Velimiroviciu PASS - 2012-04-04 PASS - 2012-04-05 PASS - 2012-04-04
Silent Update: UAC Service Simona Badau PASS - 2012-04-05 N/A N/A

Update Tests

The following are details of update verifications

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-04-04 PASS - 2012-04-05
Windows XP PASS - 2012-04-04 PASS - 2012-04-05
Windows Vista PASS - 2012-04-04 PASS - 2012-04-05
Windows 7 32-bit PASS - 2012-04-04 PASS - 2012-04-05
Windows 7 64-bit PASS - 2012-04-04 PASS - 2012-04-05
Mac OS X 10.6 PASS - 2012-04-04 PASS - 2012-04-05
Mac OS 10.7 PASS - 2012-04-04 PASS - 2012-04-05
Ubuntu 11.10 32-bit PASS - 2012-04-04 PASS - 2012-04-05
Ubuntu 11.10 64-bit PASS - 2012-04-04 PASS - 2012-04-05
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox12 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Queries

QAWANTED

Bugs: query

Need Follow-up
Full Query
ID Summary Priority Status
741179 crash in nsDiskCacheBlockFile::Write -- RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);


QA Investigating
Full Query
ID Summary Priority Status
722538 block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D -- RESOLVED
738661 crash in nmsvc from CovenantEyes P3 RESOLVED
742723 Empty bookmarks toolbar on startup due to All-in-one sidebar add-on -- RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);


Known Issues

Full Query
ID Summary Priority Status
708557 Html5 youtube context menu not persistent when right clicking -- RESOLVED
709029 The RealPlayer plugin crashes when trying to play a RealPlayer video -- RESOLVED
725296 Browser is unresponsive while Silverlight content is loading -- RESOLVED
725614 Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content P5 RESOLVED
729872 Window deactivate while in HTML5 fullscreen causes text fields to not show cursor or selection -- VERIFIED
731981 Missing steps for missing Shockwave plugin notification -- RESOLVED
734837 SHistory gets broken when navigating to and from cnn.com -- RESOLVED
735243 On HTML5 videos Esc key exits from both full screen and Highlighter -- RESOLVED
740304 "Tell Mozilla about the crash" checkbox is unchecked by default on Linux P2 RESOLVED
740321 Address bar unresponsive when previously focusing a pdf opened with Acrobat -- RESOLVED
740330 Yandex Toolbar crashes Firefox -- RESOLVED

11 Total; 0 Open (0%); 10 Resolved (90.91%); 1 Verified (9.09%);


Issues Found

Full Query
ID Summary Priority Status
742363 Crash when restarting Firefox after add-on installation and pdf document opened with Adobe Reader 9.4 -- RESOLVED
742745 The service is not uninstalled with the last build if the installation directory is deleted -- RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);


Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 5

Builds: HTTPS
Litmus: testrun, report

Checklist
  • [ON TRACK] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [ON TRACK] Bug fix verifications (10%)
  • [ON TRACK] Check with Support for top issues from previous Beta/Release
  • [ON TRACK] Check with Marcia for top-crashers spiking
  • [ON TRACK] Update tests on releasetest
  • [ON TRACK] Get final sign-off from Bob Moss
  • [ON TRACK] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-04-11
Windows XP ashughes PASS - 2012-04-11
Windows Vista ashughes PASS - 2012-04-11
Windows 7 32-bit ashughes PASS - 2012-04-11
Windows 7 64-bit ashughes PASS - 2012-04-11
Mac OS X 10.6 ashughes PASS - 2012-04-11
Mac OS X 10.7 ashughes PASS - 2012-04-11
Ubuntu 11.10 32-bit ashughes PASS - 2012-04-11
Ubuntu 11.10 64-bit ashughes PASS - 2012-04-11
Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result
Windows 2000 Simona PASS - 2012-04-12
Windows XP Paul PASS - 2012-04-12
Windows Vista Simona PASS - 2012-04-12
Windows 7 32-bit Mihaela PASS - 2012-04-12
Windows 7 64-bit Paul PASS - 2012-04-12
Mac OS X 10.6.8 VladG PASS - 2012-04-12
Mac OS X 10.7.2 VladG PASS - 2012-04-12
Ubuntu 11.10 32-bit Virgil PASS - 2012-04-12
Ubuntu 11.10 64-bit Virgil PASS - 2012-04-12

Results in Litmus

Focus Testing

The following are the detailed results of the focused feature testing. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows 7 32-bit Mihaela PASS - 2012-04-12
Mac OS X 10.6.8 Vlad PASS - 2012-04-12
Ubuntu 11.10 32-bit Virgil PASS - 2012-04-12
Windows XP Paul PASS - 2012-04-12
  1. [DONE] Rich Profile Updates Etherpad
Feature Sign-off Testing

The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.

Feature Owner Win32 Mac Linux
Media Controls Refresh Mihaela Velimiroviciu PASS - 2012-04-13 PASS - 2012-04-13 PASS - 2012-04-13
Silent Update: UAC Service Simona Badau PASS - 2012-04-13 N/A N/A

Update Tests

The following are details of update verifications

Platform Releasetest: report Beta: [ report]
Windows 2000 PASS - 2012-04-12 RESULTS - 2012-04-13
Windows XP PASS - 2012-04-12 RESULTS - 2012-04-13
Windows Vista PASS - 2012-04-12 RESULTS - 2012-04-13
Windows 7 32-bit PASS - 2012-04-12 RESULTS - 2012-04-13
Windows 7 64-bit PASS - 2012-04-12 RESULTS - 2012-04-13
Mac OS X 10.6 PASS - 2012-04-12 RESULTS - 2012-04-13
Mac OS 10.7 PASS - 2012-04-12 RESULTS - 2012-04-13
Ubuntu 11.10 32-bit PASS - 2012-04-12 RESULTS - 2012-04-13
Ubuntu 11.10 64-bit PASS - 2012-04-12 RESULTS - 2012-04-13
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox12 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Queries

QAWANTED

Bugs: query

Need Follow-up
Full Query
ID Summary Priority Status
741179 crash in nsDiskCacheBlockFile::Write -- RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);


QA Investigating
Full Query
ID Summary Priority Status
722538 block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D -- RESOLVED
738661 crash in nmsvc from CovenantEyes P3 RESOLVED
742723 Empty bookmarks toolbar on startup due to All-in-one sidebar add-on -- RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);


Issues Found

Bug Status

Known Issues

Full Query
ID Summary Priority Status
700493 Firefox Crash in nsJARChannel::OnStopRequest @ nsXULPrototypeScript::SerializeOutOfLine (Correlated to Yandex Bar and Спутник @Mail.Ru) -- VERIFIED
729907 IMDB videos lose their image and controls after restart -- RESOLVED
742363 Crash when restarting Firefox after add-on installation and pdf document opened with Adobe Reader 9.4 -- RESOLVED

3 Total; 0 Open (0%); 2 Resolved (66.67%); 1 Verified (33.33%);


Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 6

Builds: HTTPS
Litmus: testrun, report

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [ON TRACK] Bug fix verifications (10%)
  • [DONE] Check with Support for top issues from previous Beta/Release
  • [DONE] Check with Marcia for top-crashers spiking
  • [DONE] Update tests on releasetest
  • [DONE] Get final sign-off from Bob Moss
  • [DONE] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-04-18
Windows XP ashughes PASS - 2012-04-18
Windows Vista ashughes PASS - 2012-04-18
Windows 7 32-bit ashughes PASS - 2012-04-18
Windows 7 64-bit ashughes PASS - 2012-04-18
Mac OS X 10.6 ashughes PASS - 2012-04-18
Mac OS X 10.7 ashughes PASS - 2012-04-18
Ubuntu 11.10 32-bit ashughes PASS - 2012-04-18
Ubuntu 11.10 64-bit ashughes PASS - 2012-04-18
Test Failures
  • Ubuntu 11.1 32-bit /testAwesomeBar/testSuggestHistoryBookmarks - controller.assertJS: Failed for subject.enteredTitle == subject.underlinedTitle'
  • OS X 10.7 64-bit: /testManagerKeyboardShortcut.js - Selected category has been loaded

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result
Windows 2000 Simona PASS - 2012-04-18
Windows XP Ioana PASS - 2012-04-18
Windows Vista Simona PASS - 2012-04-18
Windows 7 32-bit Mihaela PASS - 2012-04-18
Windows 7 64-bit Mihaela PASS - 2012-04-18
Mac OS X 10.6.8 VladG PASS - 2012-04-18
Mac OS X 10.7.3 VladG PASS - 2012-04-18
Ubuntu 11.10 32-bit Virgil PASS - 2012-04-18
Ubuntu 11.10 64-bit Virgil PASS - 2012-04-18

Results in Litmus

Focus Testing

The following are the detailed results of the focused feature testing. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows 7 32-bit Mihaela PASS - 2012-04-18
Mac OS X 10.6.8 Vlad PASS - 2012-04-18
Ubuntu 11.10 32-bit Virgil PASS - 2012-04-18
Windows XP Simona PASS - 2012-04-18
  1. [DONE] Rich Profile Updates Etherpad
Feature Sign-off Testing

The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.

Feature Owner Win32 Mac Linux
Media Controls Refresh Mihaela Velimiroviciu PASS - 2012-04-19 PASS - 2012-04-19 PASS - 2012-04-19
Silent Update: UAC Service Simona Badau PASS - 2012-04-19 N/A N/A

Update Tests

The following are details of update verifications

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-04-18 PASS - 2012-04-19
Windows XP PASS - 2012-04-18 PASS - 2012-04-19
Windows Vista PASS - 2012-04-18 PASS - 2012-04-19
Windows 7 32-bit PASS - 2012-04-18 PASS - 2012-04-19
Windows 7 64-bit PASS - 2012-04-18 PASS - 2012-04-19
Mac OS X 10.6 PASS - 2012-04-18 PASS - 2012-04-19
Mac OS 10.7 PASS - 2012-04-18 PASS - 2012-04-19
Ubuntu 11.10 32-bit PASS - 2012-04-18 PASS - 2012-04-19
Ubuntu 11.10 64-bit PASS - 2012-04-18 PASS - 2012-04-19
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox12 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Queries

QAWANTED

Bugs: query

QA Investigating
Full Query
ID Summary Priority Status
722538 block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D -- RESOLVED
738661 crash in nmsvc from CovenantEyes P3 RESOLVED
742723 Empty bookmarks toolbar on startup due to All-in-one sidebar add-on -- RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);


Issues Found

Bug Status

Known Issues

Full Query
ID Summary Priority Status
700493 Firefox Crash in nsJARChannel::OnStopRequest @ nsXULPrototypeScript::SerializeOutOfLine (Correlated to Yandex Bar and Спутник @Mail.Ru) -- VERIFIED
729907 IMDB videos lose their image and controls after restart -- RESOLVED
742363 Crash when restarting Firefox after add-on installation and pdf document opened with Adobe Reader 9.4 -- RESOLVED

3 Total; 0 Open (0%); 2 Resolved (66.67%); 1 Verified (33.33%);


Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Release Candidate Build 1

Builds: HTTPS
Litmus: testrun, report
Mozmill: [1], updates
Build ID: 20120420145725

Checklist
  • [DONE] Functional automation
  • [DONE] Sync automation to Litmus
  • [DONE] Manual tests
  • {{ |}} Bug fix verifications
  • {{ |}} Check with Support for top issues from previous Beta/Release
  • {{ |}} Check with Marcia for top-crashers spiking
  • [DONE] Update tests on releasetest
  • {{ |}} Get final sign-off from Bob Moss
  • {{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-04-21
Windows XP ashughes PASS - 2012-04-21
Windows Vista ashughes PASS - 2012-04-21
Windows 7 32-bit ashughes PASS - 2012-04-21
Windows 7 64-bit ashughes PASS - 2012-04-21
Mac OS X 10.6 ashughes PASS - 2012-04-21
Mac OS X 10.7 ashughes PASS - 2012-04-21
Ubuntu 11.10 32-bit ashughes PASS - 2012-04-21
Ubuntu 11.10 64-bit ashughes PASS - 2012-04-21
Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result
Windows 2000 Simona PASS - 2012-04-23
Windows XP Ioana PASS - 2012-04-23
Windows Vista Simona PASS - 2012-04-23
Windows 7 32-bit Mihaela PASS - 2012-04-23
Windows 7 64-bit Teodosia PASS - 2012-04-23
Mac OS X 10.6.8 VladG PASS - 2012-04-23
Mac OS X 10.7.3 Alex PASS - 2012-04-23
Ubuntu 11.10 32-bit Remus PASS - 2012-04-23
Ubuntu 11.10 64-bit Ioana PASS - 2012-04-23

Results in Litmus

Focus Testing

The following are the detailed results of the focused feature testing. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows 7 32-bit Mihaela PASS - 2012-04-23
Mac OS X 10.6.8 VladG PASS - 2012-04-23
Ubuntu 11.10 32-bit Virgil PASS - 2012-04-23
Windows XP Ioana PASS - 2012-04-23
  1. [DONE] Rich Profile Updates Etherpad
  2. [DONE] Search Etherpad
Feature Sign-off Testing

The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.

Feature Owner Win32 Mac Linux
Media Controls Refresh Mihaela Velimiroviciu PASS - 2012-04-23 PASS - 2012-04-23 PASS - 2012-04-23
Silent Update: UAC Service Simona Badau PASS - 2012-04-23 N/A N/A

Update Tests

The following are details of update verifications

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-04-22 PASS - 2012-04-24
Windows XP PASS - 2012-04-22 PASS - 2012-04-24
Windows Vista PASS - 2012-04-22 PASS - 2012-04-24
Windows 7 32-bit PASS - 2012-04-22 PASS - 2012-04-24
Windows 7 64-bit PASS - 2012-04-22 PASS - 2012-04-24
Mac OS X 10.6 PASS - 2012-04-22 PASS - 2012-04-24
Mac OS 10.7 PASS - 2012-04-22 PASS - 2012-04-24
Ubuntu 11.10 32-bit PASS - 2012-04-22 PASS - 2012-04-24
Ubuntu 11.10 64-bit PASS - 2012-04-22 PASS - 2012-04-24
Additional Spotchecks
  • No What's New page on update - PASS (2012-04-24)
Issues

3.6.28 Major Update

Platform Releasetest: report Release: report
Windows 2000 PASS - 2012-04-22 PASS - 2012-04-24
Windows XP PASS - 2012-04-22 PASS - 2012-04-24
Windows Vista PASS - 2012-04-22 PASS - 2012-04-24
Windows 7 32-bit PASS - 2012-04-22 PASS - 2012-04-24
Windows 7 64-bit PASS - 2012-04-22 PASS - 2012-04-24
Mac OS X 10.6 PASS - 2012-04-22 PASS - 2012-04-24
Mac OS 10.7 PASS - 2012-04-22 PASS - 2012-04-24
Ubuntu 11.10 32-bit PASS - 2012-04-22 PASS - 2012-04-24

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox12 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Queries

QAWANTED

Bugs: query

QA Investigating
Full Query
ID Summary Priority Status
722538 block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D -- RESOLVED
738661 crash in nmsvc from CovenantEyes P3 RESOLVED
742723 Empty bookmarks toolbar on startup due to All-in-one sidebar add-on -- RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);


Issues Found

Full Query
ID Summary Priority Status
747858 A horizontal line is displayed on the Play button from HTML5 videos -- RESOLVED
747946 [linux] Slow rendering in Firefox with crazybugs.ivank.net web gl game -- RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);


Known Issues

Full Query
ID Summary Priority Status
700493 Firefox Crash in nsJARChannel::OnStopRequest @ nsXULPrototypeScript::SerializeOutOfLine (Correlated to Yandex Bar and Спутник @Mail.Ru) -- VERIFIED
709029 The RealPlayer plugin crashes when trying to play a RealPlayer video -- RESOLVED
725614 Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content P5 RESOLVED
729907 IMDB videos lose their image and controls after restart -- RESOLVED
742363 Crash when restarting Firefox after add-on installation and pdf document opened with Adobe Reader 9.4 -- RESOLVED

5 Total; 0 Open (0%); 4 Resolved (80%); 1 Verified (20%);


Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.