Calendar:QA TestDay:2007-05-15

From MozillaWiki
Jump to: navigation, search

<< Back to Calendar QA TestDays


When is the test day?

  • The test day will run from 13:00 UTC on {{{startmonth}}} 15th 2007 to 01:00 UTC on {{{endmonth}}} 16th 2007. The test day will start exactly when the nightly builds are available and will end 8 hours later. We realize that this may not be optimal for all our testers, so if there is great demand, we will extend the ending time.
  • Find the starting time in your timezone
  • Find the ending time in your timezone

Where do I start?

OK, I am in the channel, now what?

Let the moderators know if you have any questions or encounter any problems. If you are not familiar with our bugzilla bug tracking system, feel free to report your problems to a moderator and they can log any bugs for you.

Who will be moderating the #calendar-qa channel on the Test Days?

The moderator's nicks for this test day will be:

  • ctalbert
  • ssitter
  • Fallen
  • mschroeder
  • Andreas
  • Sebo

What build should I test?

We will be testing the latest nightly builds of Sunbird and Lightning:


Sunbird:

http://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/nightly/latest-mozilla1.8/

Lightning (for Thunderbird 3.0b2pre):

http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/latest-mozilla1.8/

What specifically will we be testing?

We need to run through as many tests as we can to perform a full regression on the RC builds.

  • Lightning BFT and FFT runs in Litmus
  • Sunbird BFT and FFT runs in Litmus
  • Sunbird locale testing (And Lightning if available)
  • Upgrade from 0.3 to 0.5 and from 0.3.1 to 0.5 testing

Some Finer Points

These are of course covered in our Litmus test cases above, but if you want some high priority places to start testing, here they are.

  • Test both Sunbird and Lightning with Local calendars (stress test, import, export, odd/arcane recurrence rules etc).
  • Test both Sunbird and Lightning with CalDav servers (Cosmo, Leopard)
  • Test both Sunbird and Lightning with WebDAV servers (Apache backends, OpenEx backends).
  • Test Lightning with the WCAP provider as a backend calendar store.
  • Test both Sunbird and Lightning with the extension for Google Calendar, keeping in mind its limitations.
  • Test the updated Look and Feel of the views. Test Copy/Paste, Drag/Drop, Drag to Create, Drag to modify, Tasks in View, etc with single and recurring events.
  • Test Lightning with the prototype Event Dialog enabled (use WCAP extension)
  • Test on Mac OS X 10.3
  • Test Alarms
  • Test printing on both Lightning and Sunbird on various OS's
  • Test Receiving invitations in Lighting with Thunderbird 2.0.0.4pre and 1.5.0.12pre as well as 2.0 from Lightning, Outlook, Apple iCal, Evolution, kOrganizer, etc
  • Test Sending invitations in Lighting with Thunderbird 2.0.0.4pre and 1.5.0.12pre as well as 2.0 to be certain they are understood in Lightning, Outlook, Apple iCal, Evolution, kOrganizer etc.
  • Test sending responses to invitations (Lightning should recognize a response but be unable to handle it further). Send responses to Lightning on the Thunderbird versions from Lightning, Outlook, Apple iCal, Evolution, kOrganizer, etc
  • Verify that these blockers are fixed -- leave a comment in each bug you verify containing the Build Identifer (Help->About on windows & Linux, AppName->About on Mac, On Thunderbird also include the Build ID of LIghtning (available below)). If the bug still occurs (i.e. it isn't fixed), note it in your comment, and let us know IMMEDIATELY in #calendar-qa.
  • Test non-mozilla timezones, work out the best understanding of the issues in bug 314339 so that it can be properly noted in the release notes (for the last time, we hope). For example, which providers are affected, to what extent.

BEFORE YOU START--BACKUP YOUR PROFILE

Though we don't expect things to go wrong, we always strongly encourage you to back up your profile and to do your testing on a "test" profile.

Backing up a profile is easy:
1. Find your profile directory:

OS Thunderbird Profile Sunbird Profile
Windows  %AppData%\Thunderbird  %AppData%\Mozilla\Sunbird
Linux ~/.thunderbird ~/.mozilla/sunbird
MacOSX ~/Library/Thunderbird ~/Library/Application Support/Sunbird

2. Copy the root of it somewhere else, for example copy the Thunderbird or Sunbird directory and everything below it.

Creating a testing profile is also easy:

  1. Launch your application with a -P option (from a terminal prompt)
  2. Create a new profile by clicking the 'Create Profile' button
  3. Click 'Next', Give it a name like "Testing Profile", Click 'Finish'
  4. Select the testing profile, and click the 'Start Calendar' (Or 'Start Shredder' in Thunderbird) button.

Remember that you will have to start the application with the -P option during testing so you can pick your testing profile

Once you've finished testing, you can delete the extra profiles and you'll be back to normal. If something goes terribly wrong (although we don't expect it to), close the application and just replace the profile with the backed up copy. That will reset the application to the way it was before you started testing.

Testing Instructions

To Run Litmus Test Cases 

Build ID Test Day
--------------------------------
OS      | Sunbird    | Lightning
Linux   | 2007051504 | 2007051503
Mac     | 2007051503 | 2007051506
Windows | 2007051505 | 2007051504
  1. Download Sunbird or Lightning from the above URLs
  2. Create your testing profile (see above)
  3. Go to Litmus.
    1. Select Run Tests, the Product is Calendar. Your buildId should match the one in the table above. Verify that it does. Sunbird's buildId is in the "About Sunbird" window. Lightning's buildId is in its description in the Add-ons Manager.
    2. Choose a Test group and subgroup to test on. If you are running the lightning-wcap extension, please run the "Prototypes" and "WCAP Calendar" tests prior to other tests.
    3. Follow the test steps and verify the function under test works as described in Expected Results.
    4. If the test case performs as expected, mark it as a "Pass". If not, mark it as a "Fail" and then enter a bug (see below).
    5. When you've finished one test, go to the next test by clicking the test case name.
    6. Once you've completed all the tests in a group, click the "Submit All Results" button at the bottom.

BUG FOUND!

  1. In the process of working with the Bugzilla or Litmus, you may find another bug. When you think you've found a bug, try to reproduce it. Restart Sunbird/Lightning and do the same steps again. The most valuable bugs are those that can be reproduced.
  2. Once the bug has been reproduced, search for it or something similar on Bugzilla. This can easily be done by searching for the error message, window name, or some other identifying feature of the bug. If you found an existing bug that is just like the bug you found, read the comments. Maybe your situation is slightly different and your input could help developers find the origin of the bug.
  3. If you can't find the bug in Bugzilla, talk about it in the channel. Maybe others can confirm your bug or find a similar bug in Bugzilla. This also gives moderators a chance to guide you to either provide more details, update an existing bug or file a new bug.
  4. If you are certain you found a unique bug and have read the Bug Writing Guidelines and Bugzilla Etiquette, go ahead and file the bug in Bugzilla. Remember to be as specific as possible.

Some other ways you can help:

  • Think of some test cases that aren't already included in Litmus (we love suggestions!).
  • Ask your friends to join you. The more people to help the better!
  • Consider looking at the Calendar ToDo list, there is much you can help with.

Where do I find Litmus and Bugzilla?

  • Bugzilla can be found at http://bugzilla.mozilla.org/
  • A list of Calendar specific bugs (without enhancements) can be found here
  • A list of already filed enhancement requests can be found here

Tips:

  • Make sure you enter the correct information for your platform, version of Thunderbird, Lightning etc. Please include the full build identifier (i.e Thunderbird 2.0.0.0 (20070326) with Lightning 0.8pre (2008020504)).
  • Be sure to read the Testing Instructions above before filing a bug.
  • Feel free to ask on IRC if you need help or have any more questions.

How do I send feedback?

Please use email us at dev-apps-calendar at lists.mozilla.org to send feedback about how the testday went, any overall thoughts about Sunbird and Lightning, and to follow up on issues found and bugs filed. We really appreciate your help today and your feedback is very valuable.

Rewards

There will be a reward for the best tester on this test day! You'll get 5 points for any non-duplicate bug that you find, as well 2 points for each Litmus test case run or for bug verified. Whoever has the most points, wins. Winners will be announced at our next Calendar QA Chat on Thursday, May 17th at 16:30UTC.

By the way, if it's worth your while to drop in to the QA Chat just for a bit, if you think you won, because if we can't find you, we can't get your reward to you!

Thank you very much for your help to make the calendar project great.

We couldn't do it without you!

Retrieved from "https://wiki.mozilla.org/index.php?title=Calendar:QA_TestDay:2007-05-15&oldid=56842"