Firefox:2.0 QA Activities

From MozillaWiki
Jump to: navigation, search

Test Plan

Strategy

  • Focus on new feeatures and develop automated tests or manual tests in Litmus
  • Ramp up from smoke tests (old regression tests and new functional tests) to FFTs (old regression and new functional) to peak with the FFTs during the Beta 2 and RC1 test cycles. Then switch back to BFTs on Windows and smoke tests for RC2, RC3 and any respins.

Milestone by Milestone plans:

Milestone Smoke test BFT FFT Focused Testing
Alpha3 XXX XXX
Beta 1 XXX XXX
Beta 2 XXX XXX
RC 1 XXX XXX
RC 2 XXX XXX
RC 3 XXX XXX
Respins XXX XXX

Old Feature Regression testing

  • This testing is accomplished with the old (FFx 1.5) Smoge tests, BFTs and FFTs.
  • We count on the unit tests run from tinderbox after the builds for a great deal of the regresion testing along
  • We also count on Community testing. Bake time on the trunk is important along with our nightly testers and the bi-weekly commnity test days.
  • We run Old BFT weekly
    • BFT Schedule and Results
    • We plan to do this until FF2 stabalizes
      • Primary platform; weekly: Win
      • Secondary platforms; alternate Intel Mac with Linux, PPC Mac every other week.
  • We run JS Test Library
  • We run Security Test Library

New Feature Functional Testing

  • New Feature Test Plans
  • Run new feature tests as code in integrated and at milestones. Generally BFT level functional tests will be ready for Beta 1. FFT level tests will be ready for Beta 2. But this is dependent on when code is ready and info is available to QA. User based tests go into Litmus. Programmatic test are checked into CVS.

L10n Testing

General strategy:

  • Verify trademark issues with info from product management
  • Thorough translation, context and layout testing with a 3rd party service for top locales and supplement with simplistic spot checks in house as needed for last minute checks, etc.
  • Test special regional distributions.

Overall Status of Locales

Detailed L10n Test Plan


Software Update Testing

General strategy:

  • Run update script to identify issues
  • Test the following matrix to identify any potential issues
  • Run the suite of Software Update tests in Litmus
  • Important Note: Alpha2 is missing Talkback

Windows Mac Linux
Alpha1 en-US en-US Fallback
Alpha2 en-US fallback
Alpha3 en-US
Beta1 (digg) One Tier 1 locale
Beta1 (official) One Tier 1 locale One Tier 1 locale fallback

Results

Extended Extension testing

There are some extension tests in the BFTs and FFTs. The extended extension testing goes beyond that to test 5 major classes of extensions:

The Criteria here is to test all the Recently added extensions, Top Extensions, Recommended Extensions. Then also test roughly the top 10 each of the Newest and Top Rated Extensions. This is a lot! This is a total of 52 extensions. We need community help with this!

The testing of this is not very detailed. The plan is to try out ~3 main features to see if the extension can be installed, used in it basic modes of operation, and uninstalls cleanly.

Anyone helping should either add their results info about what you tested (your name, feature tested, build id, Version string, pass/fail status, and notes) to the

or email results to qa@mozilla.org.

Plug-in testing

Extended Top Site testing

Profile Testing

  • Migration from Firefox 1.0, Firefox 1.5.0.x Profiles
    • Need to create a set of profiles and have them available for download
  • Profiles with History
    • Engage community help with this?
  • Profiles with the Top 10 AMO Extensions installed
  • Profiles with recommended extensions installed

Detailed Profile Test Plan

Organized Community Testing

  • Biweekly community test days
  • Weekly community bug days

Special Distributions

  • Y! CJK
  • Google CJK
  • PCHome
  • Seznam

Meeting Notes

Beta1 Feature Testing Update

Please list if feature is complete for Bon Echo beta. If not, please reference significant bugs that will not be fixed for beta1 or functionality that will not be present

  • Session Restore
    • This feature is in. Most testing is done. BC will complete his test plan by mid day Tuesday (7/11).
  • Tabbed Browsing
    • Adam needs a few more hours to finish BFTs. This should be done by COB today (7/10)
  • RSS Feed Handling
  • Anti-Phishing
  • Visual Refresh
    • Make sure that Mac popup dialog styling bug is addressed when visual refresh is done - bug 340710
  • Search Engine UI
  • In-line Spell Check
    • Not ready yet. brettw@google will let us know.
  • Microsummaries
  • Preferences Panel
    • Areas of Focus
      • Make sure new profiles and all 1.5.0.x profiles work, as well as backwards compatible profiles.
      • See bug 340677 comment 37 for specifics.
    • This is *not* in for B1. Should be there for B2.

Milestone Tasks

Beta 1 Tasks

Beta 2 Tasks

Verifications

mconnor has asked that we verify fixed crash bugs and regressions in Firefox 2.

When you verify the bug is fixed, resolve it as VERIFIED FIXED and replace the fixed1.8.1 keyword with verified1.8.1.

Tips:

  • Some bugs have been verified on 1.8.0 bug still need to be verified on 1.8.
  • If you don't see an easy way to verify that a bug is fixed comment in the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard.

Release Activities

L10n Handoff to Build

  • Axel attaches his recommended shipped-locales file to bug 344295 and requests a review of the attachment
  • QA spot checks those locales
  • Tim r+'s the attachment or adds an updated version based on the testing and r+'s it.
  • Build (Preed) sees the approved review, checks in the shipped-locales file and builds/packaged the final list of locales based on that.

Results

  • Important Note: Alpha2 builds were shipped without Talkback.' This issue was discovered and verified during testing of RC2. Therefore, users that were on the Alpha2 train who update to RC2 will not get talkback in their build.


Beta2 RC1 Test Results

RC1 Test Results

Regression Testing

  • Smoke tests
    • Win - Tomcat - PASS
      • Specific OS Version: Win2K
      • Version ID: Mozilla/5.0 (Windows; U; Windows NT 5.0;

en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0

      • Talkback Build ID 2006091818
      • Notes:
    • Mac - Marcia - PASS
      • Specific OS Version: Intel Mac
      • Version ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
      • Talkback Build ID: 2006091817
      • Notes:
    • Linux - Tracy - PASS
      • Specific OS Version: FC5
      • Version ID: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
      • Talkback Build ID: 2006091817
      • Litmus results
      • Notes:
  • BFT - Win is minimum
    • Tomcat PASS
      • Specific OS Version: Windows 2000/XP
      • Version ID: Mozilla/5.0 (Windows; U; Windows NT 5.0;

en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0

      • Talkback Build ID 20060918
      • Notes: Unclear about bring me back "anti phishing" feature
    • Marcia
      • Specific OS Version: PPC Mac
      • Version ID: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
      • Talkback Build ID: 2006091817
      • Litmus Link to results: http://tinyurl.com/mkmz5
      • Notes:
        • Did not complete a full BFT, but tested the following areas:
          • Installation, Options, prefs, General, Location Bar, RSS, Toolbar, History, PW and Form Manager, Anti-Phishing and Technical Tools.
        • No crashes observed during testing.
      • Issues:
        • Anti Phishing testcase "Get me out of here" failed, but it turned out that the behavior had changed. I fixed the test case to reflect the current behavior.
      • Bugs filed:


  • Security Test Library - Bob
  • JS Test Suite - Bob

Focused testing

Suggestions from mconnor:

  • Themes
    • This is overall area is well tested by community
    • Hover state
    • Click state on home button in help viewer
  • RSS
    • Feed preview
  • Preferences Dialog
    • Lot of late changes in this area. Test all the buttons, tabs, controls thoroughly
  • Accessibility
    • General concern
  • Tabbed Browsing
    • Throw tons of tests at the tab strip
    • There should be a warning if > 15 tabs are about to be opened at once
  • JS
    • Page compatibility
  • Overall
    • General testing
    • web compatibility testing testing

dbaron and brendan were also consulted for focus areas to test.

Test Results

Template: 

 * Tester name:
 * Platform:
 * version string:
 * Link to litmus FFT results:
 * General description (if testing was without litmus):
  • marcia - focused testing on RSS and Tabbed Browsing
  • RSS Testing: Ran set of test in Full Functionals on 9-20-06
    • Linux - Marcia -
      • Specific OS Version: Fedora Core 5
      • Version ID: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
      • Talkback Build ID: 2006091817
      • Issues Noted:
  • RSS Testing: Ran set of test in Full Functionals on 9-21-06
    • Mac - Marcia -
      • Specific OS Version: Intel Mac running 10.4.7
      • Version ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
      • Talkback Build ID: 2006091817
      • Issues Noted:
        • Added a new test case for Feed Reader First Run UI.
  • Preferences Testing - Ran Full Functionals on 9-21-06
    • Mac - Marcia -
      • Specific OS Version: Intel Mac running 10.4.7
      • Version ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
      • Talkback Build ID: 2006091817
      • Issues Noted:
  • Tomcat - Running FFT Parts for Tabbed Browsing and RSS
    • Windows 2000 SP4 /XP Sp2
      • version string: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
      • Talkback Build ID: 2006091818
      • Notes
        • Failed to open Bookmark and History Element on Background, will re-test later with my Testarray
  • Search Plugins
    • Tester name: marcia
    • Platform: Intel Mac
    • version string: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
    • Link to litmus FFT results:
    • General description (if testing was without litmus): Tested installing and uninstalling various plugins. I tested plugins from addons page as well as mycroft.
  • Themes
    • Tester name: timr
    • Platform: Windows/Mac OS
  • Accessibility
    • Tester name: jay
    • Platform: MacOS
  • Add ons - Run FFT of Add-ons tests in Litmus
    • Tester name: marcia
    • Platform: WinXP
    • version string: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
    • Link to litmus FFT results: in Litmus
    • General description (if testing was without litmus):
  • Install Extensions in 1.5 and then update to 2.0 (testing compatibility check)
    • Tester name: marcia
    • Platform: WinXP
    • version string:Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0
    • Link to litmus FFT results:
    • General description (if testing was without litmus): I tested installing a mix of extensions in a 1.5 Profile from both add-ons and extension mirror that were both compatible and incompatible with FFRC1. I then moved to FF 2 to make sure the compatibility check was done properly. PASS.

Extended Extension Testing

Tested by Tomcat on 23.September 2006 for RC 1

Extended Top Site Testing

Tested by Tomcat on 23.September 2006 for RC1

L10n Testing

See test matrix

RC1 Software Update Testing

General strategy:

  • Build Runs update script to identify issues before QA begins testing
  • Test the following matrix to identify any potential issues
  • Strong emphasis on purposely failing updates to make sure we are handling failures correctly. The script will not pick up errors in this area.
  • Run the suite of Software Update tests in Litmus
Windows Mac Linux
Alpha1 en-US en-US Fallback
Alpha2 en-US fallback
Alpha3 en-US
Beta1 One Tier 1 locale One Tier 1 locale fallback
Beta2 One Tier 1 locale One Tier 1 locale fallback

Live Testing

  • Spot checks of Software Update and all.html

RC1 Live Software Update Results

RC2.1 Testing

Path to builds: 

  http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2.0rc2-candidates/rc1/

Smoke tests

Template

  • <platform> - <tester name> - <Status PASS/FAIL>
    • Specific OS Version:
    • Version string:
    • Talkback Build ID:
    • Notes:

Results

  • Win - Tomcat - PASS
    • Specific OS Version: Windows 2000 SP4
    • Version ID: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061002 Firefox/2.0
    • Talkback Build ID 2006100202
    • Notes: PASS
  • Mac PPC - Marcia - PASS
    • Specific OS Version: 10.3.9
    • Version string: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1) Gecko/20061002 Firefox/2.0
    • Talkback Build ID: 2006100202
    • Notes: No crashes. Slow page load time on setting 4 pages as tabs, but this could be attributed to connecting via the wireless connection. Subsequent page timings did not yield any reproducable problems.
  • Mac Intel - timr - TBD
    • Specific OS Version: 10.4.8
    • Version string: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20061002 Firefox/2.0
    • Talkback Build ID: 2006100202
    • Notes:
      • Problems with images on msn.com/careers on main page and espn.com header. Reproducible on my Intel Mac, but not on two others' Intel macs.
      • Crash after testing flash plugin install, enabling/disabling adblock, and navigating between msn.com and espn.com. Not yet reproducible.
  • Linux - juan - PASS
    • Specific OS Version: Ubuntu 6.06
    • Version string: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061002 Firefox/2.0
    • Talkback Build ID: 2006100202
    • Notes: Testing those areas not covered by Aleksej. Coverage was 100% on all smoketests combined.
  • Linux - Aleksej - PASS (but see notes)
    • Specific OS Version: Debian GNU/Linux testing
    • Version string: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061002 Firefox/2.0
    • Talkback Build ID: 2006100202
    • Notes: Didn't test: standard installation, [mac only] confirm xforms not part of build, basic printing.

BFT

  • Win - Tomcat - PASS
    • Specific OS Version: Windows 2000 SP4
    • Version string: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061002 Firefox/2.0 ID:2006100202
    • Notes: No Regressions found
  • Win - Tracy - PASS
    • Specific OS Version: Win XP sp2
    • Version String: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061002 Firefox/2.0
    • Talkback ID: 2006100202
    • Litmus query: Test results
    • Notes:
      • No new regressions found.
      • I did hang once, during initial attempt, in the plug-in finder test case. However subsequent attempts worked.
  • Feel free to test other platforms!

RC2.2 Testing

Path to builds: 

  http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2.0rc2-candidates/rc2/

Smoke tests

Template

  • <platform> - <tester name> - <Status PASS/FAIL>
    • Specific OS Version:
    • Version string:
    • Talkback Build ID:
    • Notes:

Results

  • Win - tracy - PASS
    • Specific OS Version: Windows XP
    • Version ID: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061003 Firefox/2.0
    • Talkback Build ID 2006100319
    • Notes:
  • Mac PPC - tracy - PASS
    • Specific OS Version: 10.4.8
    • Version string: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1) Gecko/20061003 Firefox/2.0
    • Talkback Build ID: 2006100316


  • Mac Intel TBD - optional
    • Specific OS Version:
    • Version string:
    • Talkback Build ID:
    • Notes:
  • Linux - tracy - PASS
    • Specific OS Version: FC5
    • Version string: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061003 Firefox/2.0
    • Talkback Build ID: 2006100316
    • Notes:

BFT

  • Win PASS - Tomcat
    • Specific OS Version: Windows 2000 SP4
    • Version string: ozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061003

Firefox/2.0

    • Notes: PASS


  • Feel free to test other platforms!

Regression Testing

[marcia]

Bug to Verify

Remaining RC2 bug fixes to be verified. The idea here is to verify these bugs, particularly the one with Severity = "critical" or "blocker".

RC2 bug fixes verified

Full list of bugs fixed 9/22/06 to 9/28/06 based on bonsai query

Full list from Bonsai. These can be used to mark which ones can't be verified (due to nature of the bug or lack of reproducible test case).

  1. bug 352968 - verified
  2. bug 353160 - verified
  3. bug 352754 - verified
  4. bug 353580 - verified
  5. bug 353303 - verified
  6. bug 353734
  7. bug 353533 - verified
  8. bug 352400 - reopened
  9. bug 352782 - verified1.8.1
  10. bug 354017 - verified1.8.1
  11. bug 354017 - verified
  12. bug 325222 - verified1.8.1
  13. bug 353894 - verified1.8.1
  14. bug 354022 - verified
  15. bug 353227 - verified
  16. bug 353881 - crash, no testcase
  17. bug 353341 - not fixed yet
  18. bug 353264 - verified
  19. bug 353454 - verified
  20. bug 353897 - verified1.8.1
  21. bug 354045 - verified
  22. bug 349904 - verified
  23. bug 354099 - Vista bug, need QA
  24. bug 354147 - needs testcase
  25. bug 334110
  26. bug 353266
  27. bug 353704
  28. bug 354316
  29. bug 354034
  30. bug 354062
  31. bug 354127
  32. bug 354294
  33. bug 354334 - verified
  34. bug 354071 - verified
  35. bug 354644 - still NEW, though looks fixed
  36. bug 354541 - verified
  37. bug 354393 - verified
  38. bug 354670 - verified1.8.1

Note: Backed out or not FF2 relevant:

  1. bug 353757 - backed out
  2. bug 353388 - backed out
  3. bug 353757 - backed out
  4. bug 351610 - mail
  5. bug 226839 - mail
  6. bug 301945 - mail
  7. bug 206408 - mail
  8. bug 278096 - mail
  9. bug 353590 - mail
  10. bug 324574 - mail
  11. bug 342065 - mail
  12. bug 352801 - mail
  13. bug 354381 - mail
  14. bug 289339 - mail
  15. bug 345887 - mail
  16. bug 164733 - camino
  17. bug 340273 - camino
  18. bug 346111 - camino
  19. bug 352760 - camino
  20. bug 353964 - camino
  21. bug 354002 - camino
  22. bug 354201 - calendar
  23. bug 351168 - calendar
  24. bug 346318 - calendar
  25. bug 349788 - calendar
  26. bug 353470 - calendar
  27. bug 335462 - calendar
  28. bug 352795 - calendar
  29. bug 352862 - calendar
  30. bug 354073 - calendar
  31. bug 353567 - calendar
  32. bug 337712 - calendar
  33. bug 328576 - calendar
  34. bug 354633 - calendar
  35. bug 354465 - calendar
  36. bug 351428 - seamonkey
  37. bug 58523 - seamonkey
  38. bug 338318 - seamonkey
  39. bug 342560 - seamonkey
  40. bug 351812 - seamonkey
  41. bug 310409 - seamonkey
  42. bug 208363 - seamonkey
  43. bug 312287 - seamonkey
  44. bug 354747 - lightning

RC2 Software Update Testing

  • Testing Matrix can be found at the link below:

Locales Testing

Windows Mac Linux
de - Tomcat - PASS
en-GB -marcia
es-ES - marcia es-ES - marcia
fr-FR - juanb - PASS
ja - juanb - PASS ja-JP - marcia ja - juanb
pl - juanb - PASS pl - juanb


    • Tester : Tomcat
    • Plattform : Windows 2000 SP4
    • Locale: DE Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.1) Gecko/20061002 Firefox/2.0
    • Result : PASSED
  • Locale: es-ES
    • Tester : marcia
    • Platform : Intel Mac 10.4.8
    • Talkback ID: 2006100202
    • Version String: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; es-ES; rv:1.8.1) Gecko/20061002 Firefox/2.0
    • Result : PASSED

Top Site Testing

Plug-in testing

Template:

  • Tester:
    • Date
    • Specific OS platform:
    • Firefox Version String:
    • Plug-ins tested
    • Sites visited :

Results:

Using existing plug-ins, visit sites that use these plug-ins

Update all the typical plug-ins to the latest version and re-test the sites that use these plug-ins

  • Tester: TBD
    • Date
    • Specific OS platform:
    • Firefox Version String:
    • Plug-ins tested
    • Sites visited

Typical Plug-ins:

  • Quicktime
  • Flash
  • Java
  • (feel free to add others)

Others:

  • Flip4Mac Windows Media Plugin 2.1
  • Digital Rights Management Plugin
  • Feel free to add others

To check your plug-ins type "about:plugins" in the URL bar of Firefox.

Tester: marcia

RC3 Testing

Path to builds:

http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2.0rc3-candidates/rc1/

Smoketests

Template

  • <platform> - <tester name> - <Status PASS/FAIL>
    • Specific OS Version:
    • Version string:
    • Talkback Build ID:
    • Notes:

Results

  • Win - tracy - PASS
    • Specific OS Version: Windows XP
    • Version ID: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • Talkback Build ID 2006101023
    • Notes:
  • Mac PPC - tracy - PASS
    • Specific OS Version: 10.4.8
    • Version string: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • Talkback Build ID: 2006101022
  • Mac Intel - marcia - PASS
    • Specific OS Version: 10.4.8
    • Version string: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • Talkback Build ID: 2006101022
    • Notes:
      • Fasterfox (151049 downloads) and Tab Mix Plus (77512 downloads) are not compatible, although they are in the list of top ten extensions.
  • Linux - tracy - PASS
    • Specific OS Version: FC5
    • Version string: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • Talkback Build ID: 2006101022
    • Notes:
  • Linux - Aleksej
    • Specific OS Version: Debian GNU/Linux testing
    • Version string: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • Talkback Build ID: 2006101022
    • Notes:
      • When loading a site press Stop button while images are loading, and the statusbar won't show "Stopped". At http://www.apple.com, it shows "Waiting for images.apple.com...", at http://www.fishki.net, it shows, e.g., "Transferring data from www.webalta.com" or "Transferring data from ru.fishki.net", on www.cnn.com "Waiting for i.a.cnn.net". Didn't happen on imageshack, photosight.ru. Looks like it happens when data is being loaded from a different server.

Other Testing

  • Fun Smoketest - marcia - PASS
    • Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
  • FFT - tomcat - PASS
    • Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0

RC3 BFT

  • Win - Tomcat - PASS
    • Specific OS Version: Windows 2000 SP4
    • Version string: 2006101023 Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • Notes: Old Helpfile, should be updated with the new Firefox 2 Features - BFT ->No Crash, no show stopper -
  • Intel Mac - juanb - tried 60% - passing
    • Specific OS Version: MacOS X 10.4.8
    • Version string: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • Build ID: 2006101022
    • Notes:
  • Feel free to test other platforms!

Regression Testing

[marcia]

Bugs to Verify

RC3 Software Update Testing

  • Testing Matrix can be found at the link below:

Locales Testing

Windows Mac Linux
de - Tomcat pass
1 tier one locale -
es-ES - timr - pass es-ES - juanb - pass
fr-FR - tomcat - pass
ja - tomcat - pass ja-JP - juanb - pass ja - juanb - pass
pl - tomcat - pass pl - juanb - pass

L10N Results

Top Site Testing

  • Top 10 - Alexa Global Sites [marcia] - PASS
    • Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
  • Top 10 - Alexa Global Sites [tomcat] - PASS
    • Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
  • Top 10 - Alexa Global Sites [juanb] - PASS
    • Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0

Plug-in testing

Template:

  • Tester:
    • Date
    • Specific OS platform:
    • Firefox Version String:
    • Plug-ins tested
    • Sites visited :

Results:

Using existing plug-ins, visit sites that use these plug-ins

Update all the typical plug-ins to the latest version and re-test the sites that use these plug-ins

  • Tester: marcia
    • Date 12.October 2006
    • Specific OS platform: Intel Mac 10.4.8
    • Firefox Version String: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • Plug-ins tested
    • Sites visited
  • Tester: Aleksej
    • Date 13 October 2006
    • Specific OS platform: Debian GNU/Linux testing
    • Firefox Version String: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • Plug-ins tested
      • Java(TM) Plug-in 1.5.0_08-b03 (from a Debian repository)
      • mplayerplug-in 3.21 (mplayerplug-in.so) (from Debian)
      • Shockwave Flash 7.0 r68 (probably from Debian)
    • Sites visited :
    • Notes: mplayerplug-in-rm.so, mplayerplug-in-wmp.so and mplayerplug-in-qt.so seem to have problems here, so not tested (they show "Stopped" after some trying, tried at http://www.bbc.co.uk/ and http://www.startrek.com/).


Typical Plug-ins:

  • Quicktime
  • Flash
  • Java
  • (feel free to add others)

Others:

  • Flip4Mac Windows Media Plugin 2.1
  • Digital Rights Management Plugin
  • Feel free to add others

To check your plug-ins type "about:plugins" in the URL bar of Firefox.

Accessibility test

  • Tester : Tomcat
  • Plattform: Windows 2000 SP4
  • Build: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
  • Status : PASS -All Shortcuts worked

Extension Retest

  • Tester : Tomcat
  • Plattform: Windows 2000 SP4
  • Build: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
  • Top & Recommend Extensions not compatible to RC 3 / Firefox 2
    • Tab Mix Plus : There are working on this http://tmp.garyr.net/forum/viewtopic.php?t=3515
    • Fasterfox Bug 354933 -> No progress - Added Comment in Bug
    • PayPal Send Money Bug 354219 -> was planed for 9. October, Added Comment in Bug
    • Linkedin Bug 354213 -> was contacted , no reply so far
    • Yoono Bug 354227 -> was contacted , no reply so far

Final test checks

  • Check Firefox Bookmarks pages - placeholders as of 10-18-06 (This appears to part of the in product pages that we are going to test)
    • Help and Tutorials
    • Customize Firefox
    • Get Involved
    • About Us
    • URL`s (as example for "fr")


  • tomcats Result
    • PASS for en-US Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • PASS for de Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • PASS for ar Mozilla/5.0 (Windows; U; Windows NT 5.0; ar; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • PASS for ca Mozilla/5.0 (Windows; U; Windows NT 5.0; ca; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • PASS for cs Mozilla/5.0 (Windows; U; Windows NT 5.0; cs; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • PASS for da Mozilla/5.0 (Windows; U; Windows NT 5.0; da; rv:1.8.1) Gecko/20061010 Firefox/2.0
    • PASS for hu - Test via URL/Test Matrix
    • PASS for eu - Test via URL/Test Matrix
    • PASS for pl - Test via URL/Test Matrix
    • PASS for tr - Test via URL/Test Matrix
    • PASS for fr - Test via URL/Test Matrix
    • PASS for mn - Test via URL/Test Matrix fixed by reed
    • PASS for el - Test via URL/Test Matrix
    • PASS for ja - One 404 fixed by l10n
    • PASS for es - Test via URL/Test Matrix
    • PASS for es-AR - Test via URL/Test Matrix
    • PASS for es - Test via URL/Test Matrix
    • PASS for fy-NL - Test via URL/Test Matrix
    • FAIL for gu-IN Issues on http://gu-in.www.mozilla.com/gu-IN/firefox/about/
    • PASS for ga-IE - Test via URL/Test Matrix
    • PASS for nl - Test via URL/Test Matrix
    • PASS for pt-BR - Test via URL/Test Matrix
    • PASS for pt-PT minor Issue fixed by reed
    • PASS for sk - Test via URL/Test Matrix
    • PASS for sl - Test via URL/Test Matrix
    • PASS for sv - Test via URL/Test Matrix
    • PASS for zh-CN - Test via URL/Test Matrix
    • PASS for zh-TW - Test via URL/Test Matrix
    • PASS for nb-NO with Mozilla/5.0 (Windows; U; Windows NT 5.0; nb-NO; rv:1.8.1) Gecko/20061010 Firefox/2.0
  • Aleksej's result
    • PASS for ru - Test via URL/Test Matrix

Live Testing spot check

  • Download and install the following:
    • en-US all platforms
    • several Tier 1 locales (de, fr, ja, zh-CN, etc)
    • several Tier 2 locales
    • end to end testing (details to follow)
    • no updates to test, but you should test checking for updates manually to make sure none are offered.

robcee's testing results

  • installed the following dictionaries, testing spell-checking in forms (Mac OS X, PPC)
    • Spanish - PASS
    • German - PASS
      • didn't prompt me to restart, restart button greyed-out. Worked on restart.
    • Italian - PASS
    • English (British) - PASS
      • didn't prompt me to restart, restart button greyed-out. Worked on restart.

As Juan mentioned in IRC, some languages with special characters didn't display properly in the En-US browser. This might not be a problem if installed using the localized version from that language's country.

Aleksej's testing results

OS: Debian GNU/Linux testing

  • Builds
  • Add-ons
    • Firefox
      • bug 357874 for Extension/Theme Manager FILED
      • Something disturbing is happening at the PDF plugin installation page if you have JS disabled with NoScript and then enable it. (right-click doesn't work during the extremely long reload)
      • Web Developer: after installation, Restart button is greyed-out (bug 351819 of the Add-ons manager)
    • Extensions
      • Recommended extensions
        • Wrong version of LinkedIn Companion available - FIXED
        • Delicious Bookmarks had a bug, e-mailed to shaver
        • Cooliris Previews causes download/installation attempts on hover.
        • Forecastfox, when it tells you how to find a location code, puts long space after "Moscow" in the example for the Russian build.
        • Other recommended extensions - PASS
      • Top ten extensions
        • Fasterfox and Tab Mix Plus are not compatible with Fx2.
        • Other top-ten extensions - PASS
      • All 3 Russian spellcheck dictionaries work - PASS
    • Themes
      • Top-ten themes Silver Skin, Firefox Vista, BlackJapanMAX, macfox, macfox graphite are not compatible with Fx2.
      • Other top-ten themes - PASS
    • Search Engines - PASS
    • Plug-ins
      • Wrong version specified for Shockwave Flash - FIXED
    • addons.mozilla.org itself
      • bug 357879 on the lack of a Search field in some places FILED

tomcat`s results

OS: Windows 2000 SP4

    • Recommend Extensions (all) PASS for Windows. One wrong install Link for Linkedin Extensions. Informed shaver and #addons
    • Filed Bug 357891 Releasenote error
    • PASS for de-AT, de-DE and de-CH dicts extensions on Windows.
    • PASS for Finish dict extension on Windows
    • PASS for Hungarian (hu) dict extension on Windows
    • PASS for en-GB dict. extension on Windows.
    • Saw Bug 351819 on de - AT
    • Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 PASS for Spot Check
    • Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.1) Gecko/20061010 Firefox/2.0 PASS for spot check
    • Mozilla/5.0 (Windows; U; Windows NT 5.0; fr; rv:1.8.1) Gecko/20061010 Firefox/2.0 PASS for spot check
    • Mozilla/5.0 (Windows; U; Windows NT 5.0; zh-CN; rv:1.8.1) Gecko/20061010 Firefox/2.0 PASS for spot check
    • Mozilla/5.0 (Windows; U; Windows NT 5.0; ja; rv:1.8.1) Gecko/20061010 Firefox/2.0 PASS on spot check
    • Mozilla/5.0 (Windows; U; Windows NT 5.0; hu; rv:1.8.1) Gecko/20061010 Firefox/2.0 PASS on spot check
    • PASS on "no update" check

marcia`s results

  • Windows Vista
    • Downloaded and installed - en-US version - PASS
    • Downloaded recommended extensions
    • Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0
Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox:2.0_QA_Activities&oldid=39274"