QA/Execution/Web Testing/Goals/2012/Q1

From MozillaWiki
Jump to: navigation, search

Web QA Team Goals

Official Q1 Goals

  • Maximize and Leverage Firefox
  • Grow Mozilla
  • Establish Credible Marketplace and Identity Systems
  • Robustify and Increase Visibility/Reach of Our Tests and their Processes

Crucial Projects

These projects must be completed to achieve the above goals.

GOAL: Maximize and Leverage Firefox

  • Q1 Outcomes:
    • [DONE] Ship high-quality Firefox Flicks
    • [DONE] Have automated tests in place for static (non-published snippet) portions of the about:home page/snippet service - bug 724114

GOAL: Grow Mozilla

  • Q1 Outcomes:
    • [DONE] Audit our community contribution paths: What are we doing well, what can we improve upon? Implement two of the recommendations by end of the quarter
    • [DONE] Ensure Web QA official Community-meeting participation/attendance [attending the other community meetings, representing our team, implementing new ideas], by rotating team members
    • [MISSED] Everybody on the team leads a unique testday this quarter
    • [DONE] Rework/update the documentation on Getting Involved, on QMO (list new projects, GitHub repos, etc.)
    • [DONE] Ship high-quality Careers website
    • [DROPPED] Support a successful launch of a rebooted Air Mozilla
    • [DONE] Ship high-quality De Todos Para Todos

GOAL: Establish Credible Marketplace and Identity Systems

  • Q1 Outcomes:
    • [CARRY OVER] Ensure a performant AMO Marketplace/OWA store by load and perf-testing
      • [DONE] Evaluate at least one commercial performance/load-testing service (from SOASTA, New Relic, blitz.io, Gomez), and determine its potential fit/application
      • [DONE] Request data from Metrics for traffic visualization of AMO and if they can't provide it, find out where we can get it
      • [CARRY OVER] Create a list of possible front-end performance gotchas (culled largely from Even Faster Web Sites), go through the list with addons-devs and decide which require further action -- traded for testing BrowserQuest
    • [DONE] Work with/support the BrowserID QA team in writing BrowserID tests against http://myfavoritebeer.org/ or https://diresworb.org/, covering basic login/logout functionality
    • [DONE] Create a sanity (e.g. smoketest-level) desktop testrun for apps marketplace (this was apps-preview-dev)

GOAL: Robustify and Increase Visibility/Reach of Our Tests and their Processes

  • Q1 Outcomes:
    • [MISSED] Integrate our builds/jobs wholesale in Webdev's Jenkins
    • [DONE] Make the Firefox Beta environment the default for at least one prominent Jenkins build, and ensure it's always updated to the latest
    • [DONE] Have the ability to run mobile Selenium tests--stably--in parallel
    • [DONE] Engagement-test templates -- clean up both the output and the hard/soft-failures
    • [DONE] Test patch(es) provided for Firefox FocusManager, which are hampering our ability to run Webdriver tests quickly/reliably
    • [DONE] Basic (i.e. sanity) tests on mobile for the following projects: Mozillians, Input, and SUMO
      • log in, log out, page title check, search (positive), click-through test (navigate through two pages of a user-flow)
        • we decided to do AMO instead of Mozillians (it's under a rewrite)
    • [DONE] Come to a team decision regarding 1) whether we switch to MPL2, and 2) whether we need to include the full MPL/MPL2 license block in .py files
    • [DONE] Update https://github.com/AutomatedTester/mozwebqa-test-templates (and move it into the mozilla account) to Webdriver

Supporting Projects

Many of our projects are inter-related, and there are many supporting projects that we want to call out as they are important and cannot be entirely neglected in favor of these goals. This section calls these out.

Affiliates

  • Description of this quarter.
  • Q1 Outcomes
    • [NEW] placeholder
  • Stakeholders

AMO

  • Support a successful launch of the Developer Beta (and beyond)
  • Q1 Outcomes
    • [ON TRACK] Identify relevant performance and load metrics.
    • [ON TRACK] Create a strategy for monitoring relevant performance and load metrics.
    • [DONE] Create a mobile run of the amo tests.
  • Stakeholders

FlightDeck

  • Support the launch of the 1.0 version of the Add-on Builder
  • Q1 Outcomes
    • [DONE] Ensure and augment trunk/dev and production automation coverage, to match
  • Stakeholders

Input

  • Unclear the scope of the work for this quarter
  • Q1 Outcomes
    • [NEW] placeholder
  • Stakeholders

Mozilla.com

  • Description of this quarter.
  • Q1 Outcomes
    • [NEW] placeholder
  • Stakeholders

MDN (Mozilla Developer Network)

  • Description of this quarter.
  • Q1 Outcomes
    • [NEW] placeholder
  • Stakeholders

Mozillians

  • Description of this quarter.
  • Q1 Outcomes
    • [NEW] placeholder
  • Stakeholders

Socorro

  • Description of this quarter.
  • Q1 Outcomes
    • [NEW] placeholder
  • Stakeholders

SUMO

  • Need more info on how continuous deployment is taking shape for SUMO this quarter
  • Q1 Outcomes
    • [NEW] placeholder
  • Stakeholders