Firefox:1.5.0.x:Test Plan

From MozillaWiki
Jump to: navigation, search

General Firefox 1.5.0.x Test Plan

Bug Verifications

Identify bugs to be verified

  • Triage fixed bugs and categorize
    • Focus on security and critical bugs first
    • Add "testcase" keyword for those bugs that have reproducible steps and/or attached testcase(s)
    • Identify difficult to reproduce bugs that do not have a testcase
      • add [need testcase] in status whiteboard.
      • request more info from reporter, developer, or anyone interested in the bug
    • Spot check "no verification necessary" bugs
      • Usually internal architecture changes with no externally visible effects
      • Quick code inspection and checkin verification with bonsai

Confidence metrics and sign-off requirements

  • Security bug verifications: 100%
  • Critical bug verifications: 100%
  • Non-critical bug verifications will be done on a case by case basis, depending on developer feedback and community testing.

QA resources

  • MoCo QA is responsible for:
    • Security and critical bug verifications
    • Some non-critical bugs and special test cases
    • Providing bug lists to the community and pointing people to the wiki page
  • Mozilla Community tasks:
    • Test and help verify fixes for non-security bugs
  • Target markets and Mozilla partners:
    • IBM for enterprise deployment functionality
    • Mozilla Japan for jp-critical issues

Regression Testing

  • Smoke Tests
  • BFT on all three platforms
  • Java Script
  • Top Sites (See FFT)
  • Key extensions
  • Performance
    • Gross memory usage
  • Security Test Library tests

tomcats results

Plattform: Windows 2000 SP4 / XP SP2

  • Smoketests
    • PASS for 1508 en-US Firefox Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.8) Gecko/20061023 Firefox/1.5.0.8, all tests passed, results in litmus
    • PASS for Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.8) Gecko/20061024 Thunderbird/1.5.0.7 ID:2006102408
    • PASS for version 1.5.0.8 (20061025) Tbird

Install Testing

  • Test en-US after initial packaging.
  • Test locales after locale packaing.

L10n Testing

Extension Testing

Update Testing

This is the testing of the partial and full update .mar files for the various locales once the main downloads are released.

Community Testing

Configuration Testing

We need to describe our configuration testing approach better. For now, we are generally testing these OS's:

  • Windows XP
  • MacOS 10.3, 10.4
  • Linux Ferdora Core 4

But we need to consider our system requirements statements. I have seen at least three versions. Here is one: