QA/Test Automation/2010-10-20

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #mozmill for backchannel

Last weeks action Items

• Anthony: [DONE] Send link to the failure spreadsheet for the overview of failed tests
• Geo: [CARRY OVER] Generate wiki page to sum up tab candy objective
• Geo: [DONE] Coordinate integration of the memory into qa-horus and qa-set
• Henrik: [MISSED] Talk with John O'Duin about specifics of the test environment
• Henrik: [DONE] Talk with Clint if mozmill-dev mailing list can be used for QA Test Automation activities
  • Yes we can use this list for our Mozmill tests topics. It will result in more openness and we can combine all efforts around Mozmill.
• Henrik: [NEW] Find a way to get a notification for builds on FTP for Mozmill
  • Asked on bug 588398 but no reply yet
• Matt: [DONE] Action items from release tasks meeting? Need to notify release leads that they're responsible for coordinating resources.

Goals Overview

• QA Q4 goals

Projects

Mozmill (Henrik)

• Mozmill 1.5.1 delayed (first builds expected today/tomorrow)
  • Lot of [bug fixes] from Henrik (including page load events, menu API, common js, click position)
  • Once builds have been pushed to pypi we have to upgrade our qa boxes and push the [needs-mozmill-1.5.1] patches
  • We have to check all platforms carefully and report bugs immediately.
  • We have to verify the remaining fixes. Anyone?

Test creation / shared modules

• General (Henrik)
  • Updates for CommonJS will land in 3 steps, see bug 602365. Everyone of us should take 5 test subgroups and create patches.
  • We will have to continue to work on the remaining test failures
  • We should investigate how much efforts we have to put in cleaning up the tests when seeing the shared module refactoring approaching this year. It will completely change the tests in either way. See Henrik's [PoC].
• DOM Walker + L10n API (Adrian)
  • Both are on review - see: bug 587819 for DOM Walker and bug 604411 for the L10n API
• Redesign Shared Modules (Geo)
• Test Refactoring (Anthony)
  • Personal project to clean up formatting and style inconsistencies across all tests
  • bug 604703 is tracking this project
  • Waiting on bug 602365 (CommonJS) to land so we don't conflict

Local Test Data (Anthony)

• Divided remaining work into 2 phases:
  • Phase II:
    • Tests we can make local without major Mozmill dependencies
    • bug 604708 is tracking this project
    • Waiting on bug 602365 (CommonJS) to land so we don't conflict
  • Phase III
    • Tests which depend on unavailable Mozmill dependencies
    • bug 604716 is tracking this project
    • Probably not doable until at least Mozmill-2.0
• This project should be given low priority because it's not a huge win (only 4 tests will be made local this quarter under phase II)

Broken Tests

• General
  • Mozmill 1.5.1 will solve a lot of existing failures. Needs reinvestigation later this week.
• Firefox 4.0 (Geo)
• Firefox 3.5/3.6 (Anthony)
  • bug 604718 is tracking this project
  • created spreadsheet to categorize current failures
  • waiting until we have a couple days of results with Mozmill-1.5.1 before re-evaluating failures and prioritizing work

Update Tests (Henrik)

• Software update tests are collecting meta data as an array bug 604370. Have to finish up the automation script for the wiki output first before it can land.

MozMill Crowd extension (Henrik)

• Will continue next week - highest priority, need a demo for the meetup in London

Add-ons Testing (Henrik)

• None, awaiting Mozmill 1.5.1 for further community outreach.

Automation / Infrastructure (Henrik)

• 32GB RAM have been inserted into QA lab machines. Now they are running really smooth.

Dashboard (Henrik) staging server

• None

Personal

Henrik

• Last two weeks
  • [DONE] Make sure our requested / needed features will land for Mozmill 1.5.1
  • [DONE] Update Software update tests to collect meta data as array
  • [DONE] Update shared modules to use Common JS
  • [MISSED] Continue work on the crowd extension
  • [MISSED] Prepare proposal for my talk at the Selenium meetup in London
  • [DONE] [Setup a PoC] for the shared module refactoring
• Next two weeks
  • [CARRY OVER] Continue work on the crowd extension
  • [CARRY OVER] Prepare talk for the Selenium/Mozmill meetup in London

Anthony

• Last two weeks
  • Develop action plans for this quarter:
    • [DONE] Local Data Phase II
    • [DONE] Refactoring Tests Phase I
    • [DONE] Branch Failures
• Next two weeks
  • If/when Mozmill-1.5.1 lands
    • Re-evaluate branch failures, prioritize, and file bugs
  • If/when CommonJS lands
    • File bugs to begin implementing Local Data Phase II
    • File bugs to begin implementing Test Refactoring Phase I
  • Make sure above projects have sections on the Automation Project Wiki

Al

• Last two weeks
• Next two weeks

Geo

• Last two weeks:
  • Bug fixes for 4.0 tests (!!)
  • At least one more test (non-AwesomeBar) for the refactor examples
  • Distillation of examples to templates/instructions
  • Starting "daily standup" emails sometime this week. Hope for full team participation once we see the usefulness.
• Next two weeks:

Aaron

• Last two weeks:
  • Presently working on Add-on tests module revisions that make use of the new Add-on API
  • Fixing up 4.0 branch broken test failures
  • Work with Anthony in preliminary action steps and beginnings for test refactoring and local data, phase II
• Next two weeks:
  • Work on broken test failures under 1.5.1
  • Pending Mozmill 1.5.1 release
    • Kick off local data phase II project
    • Test refactoring phase I project

Adrian

• Last two weeks:
  • [MISSED] implement the L10n tests for the update window
  • bring the 2 shared-modules and 4 tests through the reviews
• Next two weeks:
  • L10n tests for the update window
  • ask for review of the L10n tests after the shared-modules do get a positive review

Contributor Status

• Name

Roundtable

General

• Found several major regressions from compartments landing (total Mozmill breakage, XPathEvaluator, and a crash)!!

Issues

Meeting Notes

• Goals
• Others