Mozilla QA Community:1.5.0.7 Community Test Day

From MozillaWiki
Jump to: navigation, search

Welcome to the Firefox/Thunderbird 1.5.0.7 Community Test Day!!

Thank you for participating in this test day. The success of Firefox and Thunderbird is heavily dependent on community members like you. We appreciate your support!!

When you are done, please send feedback to [1]. We are very interested to hear about how the test day went, any overall thoughts about Firefox and Thunderbird 1.5.0.7, and roll up of issues found and bugs filed. We are also looking for ways to get more people involved, so any ideas or suggestions on how to make future events more interesting and fun are welcome too! :-)

The official time for the test day is Friday, September 1, 2006, from 7am - 5pm PDT, but anyone can jump on Litmus at any time on any day!

For more information about this release, please check out the Firefox/Thunderbird 1.5.0.7 Community Wiki. You will find the current schedule, test plan, and specific areas you can help us test.

FAQ

Where do I start?

Ok, I am in the channel, now what?

  1. Let the moderators know if you have any questions or encounter any problems. If you are not familiar with our bugzilla bug tracking system, feel free to report your problems to a moderator and they can log any bugs for you.

What build should I test?

We will be testing the latest Firefox and Thunderbird 1.5.0.7 candidate builds:

Firefox: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/1.5.0.7-candidates/rc3/

Thunderbird: http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/1.5.0.7-candidates/rc2/

What needs to be tested?

The easiest thing to do...

Try running through any number of Basic Funcational Tests (BFTs) that are available through Litmus, our test tracking system.

For those that want to do a bit more exploring...

You can do some exploratory testing around our Fixed Bugs. There are plenty of them, so pick a few and then try:

  • Reading through the bug comments to get a better understanding of the problem
  • Try any testcases in the bug to see if they work for you or create your own simplified testcase or steps to reproduce.
  • Find features, components, areas, etc to test that might be effected by the fix and verify that everything works as expected.
  • If you want, you can leave a comment in the bug, or just include your results in the notes you send to us.

We have made changes or fixed bugs/regressions in parts of the code that deal with the following areas, so you should try to test around them to make sure everything still works as expected:

  • SSL certificates and websites (try going to "secure" websites to make sure everything works as expected. One site that can point you to a few more is https://www.verisign.com/, just check the customers link).
    • Financial sites
      • any banking site - try testing any online bank site that you use, ensure that it works correctly
      • any shopping site - go to your favorite online shopping site and log in to your account. Try adding things to your online cart, checking out, etc.
    • This may also effect secure gmail accounts. Try using https://gmail.google.com and see if it works correctly
    • Any other online mail accounts (yahoo, etc)
  • Websites with fancy table layouts or other unique structure. Do a Google search for layout testing and you might be able to find some test suites to run to make sure Firefox renders pages properly. You can also compare websites in Firefox to what they look like in IE, Opera, or Safari (99% of the time they should all look pretty much the same).
  • We have made a change to how Java is run on the Mac. If you are using a Mac browse any sites with Java or Java applets. Ensure that they load and work correctly
  • There have been a couple of tweaks that may effect eBay. If you have an eBay account login, place bids, check-out or do any other day-to-day activities
  • Browse your favorite sites and make sure your experience is the same as with 1.5.0.6.
  • Take a look at the Fixed Bugs list and pick a few bugs to explore and test around.

If you decide to test on your own, please try to make note of any bugs you have looked at, any testcases you have run, and any other info about your testing. When you're done, please email us at [2] with your results.

If you discover a bug or regression that you feel is critical, please let us know on the #testday IRC channel ASAP!

Where do I find Litmus?

  • Go to Litmus
  • log in and then select run tests
  • You can do a smoketest or a Basic Functional Test, it is entirely up to you!
  • Talk to a moderator if you need help.
  • Please make sure to choose the correct platform when you start your run.

Who will be moderating the #testday channel?

The moderator's nicks are:

  • tracy
  • bc
  • timr
  • marcia
  • jay
  • davel
  • juanb
  • robcee
  • alice
  • coop
  • ispiked

How do I file bugs?

First of all, let one of the moderators know so they can check if it is already a known problem. Then they can guide you to either provide more details, update an existing bug or file a new bug. If filing a new bug, please do a quick check to see if the bug already exists. This can easily be done by searching for the error message, window name, or some over identifing feature of the bug.

How do I send feedback?

Please use the [3] to send feedback about how the testday went, any overall thoughts about Firefox or Thunderbird 1.5.0.7, and roll up of issues found and bugs filed. We really appreciate your help today and your feedback is very valuable.

Results

Total Test Results By Locale

Locale # Results
en-US 236

Total Test Results By Platform

Platform # Results
Windows 153
Linux 32
Mac 51

Total Test Results By Result Status

Result Status # Results
Pass 226
Fail 8
Test unclear/broken 2

Total Test Results By Subgroup

Subgroup # Results # Testcases in Subgroup
Firefox:1.5 Basic Functional Tests (BFTs):Installation 6 3
Firefox:1.5 Basic Functional Tests (BFTs):Migration 2 1
Firefox:1.5 Basic Functional Tests (BFTs):Menu Bar 8 10
Firefox:1.5 Basic Functional Tests (BFTs):Help 12 6
Firefox:1.5 Basic Functional Tests (BFTs):Page Controls 4 4
Firefox:1.5 Basic Functional Tests (BFTs):Toolbar Customization 5 5
Firefox:1.5 Basic Functional Tests (BFTs):Location Bar & Autocomplete 3 3
Firefox:1.5 Basic Functional Tests (BFTs):Search 3 3
Firefox:1.5 Basic Functional Tests (BFTs):Printing 3 3
Firefox:1.5 Basic Functional Tests (BFTs):Bookmarks 15 15
Firefox:1.5 Basic Functional Tests (BFTs):History 8 4
Firefox:1.5 Basic Functional Tests (BFTs):Extensions 3 7
Firefox:1.5 Basic Functional Tests (BFTs):Security and Certs 8 3
Firefox:1.5 Basic Functional Tests (BFTs):Import 1 1
Firefox:1.5 Basic Functional Tests (BFTs):Plug Ins 2 1
Firefox:1.5 Full Functional Tests (FFTs):Installation 11 6
Firefox:1.5 Full Functional Tests (FFTs):Menu Bar 8 10
Firefox:1.5 Full Functional Tests (FFTs):Help 7 7
Firefox:1.5 Full Functional Tests (FFTs):XMLHttpRequest 5 2
Firefox:1.5 Full Functional Tests (FFTs):RSS 4 4
Firefox:1.5 Full Functional Tests (FFTs):DOM Inspector 2 2
Firefox:1.5 Full Functional Tests (FFTs):Page Info 6 3
Firefox:1.5 Full Functional Tests (FFTs):Top Sites 8 8
Firefox:1.5 Full Functional Tests (FFTs):View Source 5 5
Firefox:1.5 Full Functional Tests (FFTs):Security 1 1
Firefox:1.5 Smoketests:Install and Startup 12 4
Firefox:1.5 Smoketests:Uninstall 6 2
Firefox:1.5 Smoketests:Functionality 78 26
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - XMLHttpRequest 2 3

Top Testers

Tester # Results
tracy 96
mkairys 76
Tomcat 45
LonelyBob 19

Result Status Breakdown By Tester

Tester Pass Fail Unclear
LonelyBob 18 0 1
Tomcat 41 3 1
mkairys 74 2 0
tracy 93 3 0