Auto-tools/Meetings/2012-12-10

From MozillaWiki
Jump to: navigation, search

Goals Progress

Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.

  • Ensure B2G has all test automation available to prepare for launch
    • [ON TRACK] Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:
      • [DONE] Mochitest-plain (emulator first, panda second)
      • [SKIPPED] Mochitest-chome (emulator first, panda second)
      • [DONE] Reftest (emulator first, panda second
      • [ON TRACK] xpcshell (panda)
      • [DONE] webAPI (emulator only
      • [ON TRACK] webQA - gaia front end tests emulator and panda
      • [ON TRACK] gaia dev integration/unit tests emulator and panda
    • [ON TRACK] Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng
    • [ON TRACK] Ensure all webAPIs testable through the emulator have comprehensive automated tests
    • [ON TRACK] Continue to aid QA in creating automated tests for B2G
  • Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.
    • [ON TRACK] Have APC.io (ARMv6)boards smoketest ready - rooted, working, plugged into Autophone
    • [ON TRACK] Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations
    • [ON TRACK] Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing
      • Version 1.1 released. Mozharness mozpool script complete.
    • [ON TRACK] Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.
  • Create and deploy performance automation for desktop, fennec, and B2G product benchmarks
    • [ON TRACK] Create Eideticker tests for B2G on panda boards and run them once a day
    • [ON TRACK] Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis
      • Discovered two problems with the talos production data collected in datazilla, see datazilla section below for description.
    • [ON TRACK] Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions
  • Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.
    • [DEFER] Deploy 4.2 version of Bugzilla
      • Deferred due to conflict with B2G.
    • [ON TRACK] Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.
    • [ON TRACK] Complete the 'Sandstone' CSS update
    • [DEFER] Rework browserID <--> BMO integration
      • Deferred due to B2G-related distractions.
    • [ON TRACK] Deliver user and product dashboards
    • [DEFER] Deploy tracking flags extension database optimization
      • Deferred due to conflict with B2G.
  • Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem
    • [DONE] Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL
    • [ON TRACK] Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures
      • See Sheriffing/TBPL/War-on-Orange below for progress this week. Overall progress: Smartsheet.
    • [DONE] Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.
    • [ON TRACK] Make orange factor update more than once a day to help make it easier to see changes over time
    • [ON TRACK] Assist with getting Datazilla's UI integrated with TBPL, phase 1
      • See Sheriffing/TBPL/War-on-Orange below for progress this week. Overall progress: Smartsheet.
  • Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.
    • [AT RISK] Re-organize and improve the automated testing documentation at MDN
    • [ON TRACK] Develop first round of mochitests for WebRTC
    • [ON TRACK] Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage

Highlights

  • [davehunt] We're hosting a Firefox OS Automation Testday this Friday. Come along and help to contribute some automated tests. You might want to prepare your machine in advance (guides can be found in the etherpad) if you don't have a device or emulator ready.
  • [davehunt] b2gperf (time to paint) tests running for Phone, Messages, Browser, and Settings apps on Unagi device in Jenkins (requires MV VPN). Reporting to DataZilla is in progress.

In-Depth Project Discussion

Pick one project/presenter per week who will give a detail presentation on their project.

The Overview

Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!

AutoPhone/Noah's Ark

Bughunter

Bugzilla

  • no push due to work weeks and travel (with both bmo and IT teams)
  • [glob] updates to the comment tagging facility
  • [glob] more performance related work
  • [glob] minor code changes and admin work

Eideticker

  • [wlach] WIP to make tests/reporting more flexible (for B2G dashboard, if ever appears)
  • [wlach] Spent most time isolating/debugging panda issues which are preventing eideticker from working reliably on b2g.

Firebug Testing

Jetperf

Marionette

  • [DBurns] Handling blockers that MDas was previously doing as she tames the the pandas

Mobile Automation

  • [ahal] Almost ready to have B2G xpcshell tests turned on for core branches, hopefully by eod today
  • [ahal] B2G mochitests permissions issue solved by David Clarke, just waiting on gaia change to land

Mobile Evangelism Site-Scraping Tool

Mochitest

MozBase

Mozmill

MozTrap

  • [camd] - Finished up with MozTrap 1.3.1
    • show all versions of cases in manage/cases screen, not just latest
    • able to delete just a single version of a case
    • new help links to markdown syntax where applicable
    • no default of product version for new cases so it's more deliberate
  • [camd] implementing global filter settings (at the user session level)

New Graphserver (Datazilla)

  • [jeads] The build id coming in the JSON data is not unique across different PGO and Non-PGO Talos branch definitions. There may also be additional branch combinations where the build id is not unique. This corrupts the correct branch association with the test data. The issue can be fixed by creating an additional composite key.
  • [jeads] There's a set of branch names coming from Talos (Mostly Non-PGO branches) that have no match in the list of branches displayed in tbpl. This means no parent metrics data will ever be found for a given test for these branches.

Open Web Apps

Peptest

Powerball

Pulse

Signal From Noise

Speedtests

Systems

Talos

Sheriffing & Trees

Treestatus

Tinderboxpushlog

War on Orange

  • [edmorley] Another big push this week to lower our OrangeFactor. Our #1 orange by far at the moment is still the Android NSS shutdown crash, but having difficulty getting dev interest :-(

Web QA

Automation Development

  • [rwood] Submitted updated patch for WebProximitySensor tests (bug 811583)
  • [rwood] Submitted patch for WebMobileConnection.data tests (bug 811580)
  • [rwood] Test reviews for our tests and reviews of some test changes by development
  • [rwood] Patch coming today for Screen Orientation API test (bug 813777)

Upcoming Events

Standing weekly meeting related to our projects should go here

Round Table

  • Could we talk about the proposed mozprocess API change? So folks can discuss what we are planning there (jhammel, ted, wlach, one of you perhaps?)
  • Bc's logging post - how do we want to move forward with that, is it a step toward full structured logging? What's our vision there?


Take notes here and copy and paste them to this section afterwards.

Notes

Holidays and Trips

Next week MC

Misc

Find something you think is cool, interesting, funny, or exciting and put it here

Retrieved from "https://wiki.mozilla.org/index.php?title=Auto-tools/Meetings/2012-12-10&oldid=492946"