QA/Firefox TDC

From MozillaWiki
< QA
Jump to: navigation, search

Overview

The Firefox TDC QA team is in charge of the Firefox browser -- both Desktop, (Windows, Linux, Mac) and Mobile (Android). Our duties include qualifying builds prior to a maintenance or milestone release. Other responsibilities include, but are not limited to:

  • Building productive internal/external working relationships.
  • Participating in the user story (Product) priority setting
  • Triaging bugs using a triage strategy
  • Creating test strategy and test plan for Firefox features
  • Running smoke tests and integration tests - automated and manual

Prior to a release, we perform ongoing tasks on the development branches to ensure no critical issue or regression gets uplifted as we merge changes to the next branch. e.g.,

  • Confirm new unconfirmed bugs
  • Verify bugs on development branches like Nightly or Aurora
  • Interact with developers to help them test the features they develop
  • Perform exploratory testing on new features
  • Write test cases in MozTrap or Google Doc (We're moving all test cases to TestRail)
  • File and track new crasher bugs as they appear in crash-stats

Meetings (Ongoing projects)

Content Handling Enhancement

  • Type: Weekly meeting
  • Time: Monday@ 3PM-4PM (UTC+8)
  • Meeting minutes
  • Attendee:
    • EPM: Wesly Huang
    • UX: Carol Huang, Bryant Mao, Morpheus Chen
    • FE:Sean Lee, KM Lee
    • PF: Alphan Chen, Eden Chuang, Will Wang
    • QA: Cynthia Tang, William Hsu
  • Type: Stand-up meeting
  • Time: Wednesday@ 11am~11:30am, Friday@ 11am~11:30am (UTC+8)
  • Attendee:
    • EPM: Josh Cheng, Wesly Huang
    • FE:Sean Lee, KM Lee
    • PF: Alphan Chen, Eden Chuang, Will Wang
    • QA: Cynthia Tang, William Hsu

Control Center - Notification

  • Type: Weekly meeting
  • Time: Irregular
  • Attendee:
    • EPM: Vance Chen
    • FE: Fred Lin, Ricky Chien
    • QA: Cynthia Tang, William Hsu

Date/Time Input type

  • Type: Weekly meeting
  • Time: Thursday@ 3:30pm~4pm (UTC+8)
  • Attendee:
    • EPM: Wesley Huang
    • FE: Tim Guan-tin Chien, Scott Wu
    • PF: Hsin-Yi Tsai, Jessica Jong
    • UX: Harly Hsu, Morpheus Chen, Tina Hsieh
    • QA: Cynthia Tang, William Hsu

Fennec Audio Controls Enhancement

  • Type: Weekly meeting
  • Time:
  • Attendee:
    • EPM: Rachelle Yang
    • Developer: Alastor Wu
    • QA: Cynthia Tang, William Hsu

Fennec Stability and Performance Enhancement

  • Type: Weekly meeting
  • Time:
  • Attendee:
    • EPM: Rachelle Yang
    • Developer: John Lin
    • QA: Cynthia Tang, William Hsu

Team Members and Assignments

  • Manager: Al Tsai
  • QA: Cynthia Tang
  • QA: William Hsu

SoftVision QA Team

SoftVision QA Team

Community members

Feature Owners

Major features in Firefox are owned by a primary QA contact who is responsible for ensuring the proper testing of those features.

We encourage community members to try specializing in a particular Firefox feature so they gain knowledge in depth! Contact the feature owner if you would like to be a tester on their team.

Here's our current list of Feature Owners.

Planning

Content Handling Enhancement

Control Center - Notification

Date/Time Input type

Fennec Stability and Performance Enhancement

UX/QA Document

Content Handling Enhancement

Control Center - Notification

  • Test plan:
  • Test case:

Date/Time Input type

Fennec Stability and Performance Enhancement

Release Owners

Release test plans

Automation

Test automation!

The Firefox Automation team is a group of passionate and open minded people working on automation for Firefox. We are spread around the world, but we have one collective goal to empower automation even more in the Mozilla project. If you want to get in contact with one of us please check the table below for more information.

https://wiki.mozilla.org/QA/Automation

Stability

The Crashkill or Stability team! https://wiki.mozilla.org/CrashKill

Collaboration model (Draft)

Softvision and TDC QA

Current feature release cycle (Firefox) is similar to the process below. EPM, engineer, and UX person are separately in charge of planning, development, and figuring out UX spec in development stage; In the meanwhile, Engineering QA is working on test plan and test case. After all patches land on Nightly, Engineering QA will be the first gatekeeper to verify features, and then hand over feature testing to Release QA if the feature passes the exit criteria. Before branching out, sign-off mechanism is necessary for Nightly, Aurora, Beta, and Release. Therefore, release manager/engineer are taking care the tree health and watching verification/validation result in different branches.

QA Collaboration Model.jpg

Above is current release process.

Problem:

The QA/QC process may be changed and insufficient for current organization because organization structure was slightly changed. I believe that you also concerned about Desktop Firefox code quality. Because no matter who implements the feature, the code is always merged into the same Firefox branches.

Proposal:

Therefore, I was wondering if we can adopt the proposal below to let QA/QE process keep the same shape. From TDC QA perspective, we would like to take following actions

  1. TDC QA takes over QA communication effort in TDC including UX spec review and code review.
  2. TDC QA takes over testing documentation including test strategy, test plan, and testing schedule.
  3. TDC QA provides testable feature/code for softvision QA team, and Softvision QA team helps TDC to complete feature testing.

Here are the pros and cons of the proposal.

Pros

  • Overcome time zone issue because EPM, UX, Dev, and QA are in the same region.
  • Softvision QA can save more communication effort, and then do more research or perform more tests.
  • Test result can be cross-referenced if TDC QA is involved in testing.

Cons

  • If TDC takes many Firefox features, loading and effort of TDC QA will increase.
  • TDC QA is short-handed for on-demand request because it is hard for TDC to extend QA resource.

Mitigation Plan for risks

  • TDC QM needs to watch the QA resource.
  • TDC EPM needs to control the release feature.

Feature Tracking & Handover

Desktop Firefox

  1. The Desktop team uses AHA to monitor new features, however in their case it’s not as reliable as it is for Fennec. Some features can be found here, but the list is far from being complete, or even accurate in some cases.
  2. To find other new features the team does a combination of the following:
    1. Monitor AHA for updates on the features tracked there
    2. Monitor the list of bugs with the “feature” keyword – found here: https://wiki.mozilla.org/Features/Release_Tracking
    3. Monitor Beta pushlogs for major changes that we think should be tracked as features
  3. The best ways to let the Desktop QA team know about a new feature that needs testing are:
    1. Add the “feature” keyword to the meta bug, and make sure you have a target milestone set
    2. Send an email to: florin.mezei@softvisioninc.eu, and/or andrei.vaida@softvisioninc.eu, and/or cornel.ionce@softvisioninc.eu

Tracking tools used by the Desktop QA team:

@ Test Cases

  1. The team hasn’t used MozTrap for about 1 year now
  2. The team has (until recently) used Googledocs to store and run tests… see some examples below:
  3. Note that same as the Android team, Desktop has also just moved to using Test Rail as our Test Management tool, so all these tests will be moved here: https://testrail.stage.mozaws.net/index.php?/projects/overview/17 (note that VPN is needed to access this link)

@ Features

  1. The team tracks the testing status for various features in a googledoc here: https://docs.google.com/spreadsheets/d/1Rn-F3Kg_1_VznIxxXkAGGL8mVMSAdamZZI4f1O2r8HA/edit?pli=1#gid=809147728
  2. The status is also tracked for each individual Beta version – e.g. https://wiki.mozilla.org/Releases/Firefox_49/Test_Plan/Beta/2#Features

@ Individual features

  1. They have dedicated wiki pages

Fennec

The AHA link that you’ve mentioned is indeed the one that the team uses to track Android features (their status, target milestone, requirements, and others). While AHA is used to convey info on various features, it is not used to store or manage test cases. It is mainly a tool for developers to let everyone see what the planned features are, and their status.

For tracking purposes our QA team uses various systems:

@ Test Cases

  1. The team has used MozTrap as the Test Management System… see for example the following test sets:
  2. Note that we have just moved to using Test Rail as our Test Management tool, so all these tests will be moved here: https://testrail.stage.mozaws.net/index.php?/projects/overview/13 (note that VPN is needed to access this link)

@ Features

  1. The team tracks the testing status for various features in the individual version pages:

@ Individual features

  1. They have dedicated wiki pages
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Firefox_TDC&oldid=1146935"