Auto-tools/Meetings/2012-08-27

From MozillaWiki
Jump to: navigation, search

Goals Progress

Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.

  • [ON TRACK] Support the B2G and platform team by moving existing automation to that platform as well as expanding automation on B2G and B2G devices for basecamp deliverables.
    • [mihneadb] SUTAgent can be now built inside B2G
    • [mihneadb] SUTAgent can now start on boot
    • [jgriffin] Will try turning on Marionette tests in TBPL on desktop Firefox this week; will help prevent accidental Marionette breakage in B2G testing
  • [ON TRACK] Expand Mobile testing platforms for B2G and Fennec Native to include user devices and NEON Chipsets (panda boards).
  • [ON TRACK] Establish new Datazilla system as a useful tool for capturing and diagnosing talos performance regressions on a per-push basis, as well as capturing data for network and B2G performance tests.
    • NOTE: This is largely a change to the data harvesting and analysis side of the Talos system (even though some changes are required to the test harness and tests themselves). We will not be replacing existing Talos data harvesting system in Q3. By the end of Q3, we want to have the new performance data harvesting and analysis system Datazilla to a point at which it can shadow the old system and we can use Q4 to verify that the new approach captures just as many regressions as the old system does. Potential timeframe for conversion of the old system to the new Datazilla system is in Q1 2013 (earliest "thumb in the wind" estimate).
  • [ON TRACK] Upgrade BMO to 4.2, deploy tracking flags optimizations, UX improvements (comment tagging, mid-air resolution)
  • [ON TRACK] Make TBPL More Orange aware to make it easier to file intermittent orange bugs, and so that we can more adequately track (and therefore address) issues around android-related and memory leak intermittent failures
    • Have been working on issues that are causing more day-to-day aggravation (/time loss) than filing oranges the last week (eg infra load patches, tree status, anti-orange work given after today ESR17 orange will be with us until December 2013 etc), so apart from a few more discussions on implementation, didn't have any time to work on this around sheriffing. Making tree status live was one of the other proposed sheriffing goals for this quarter - can we just add it retrospectively, seeing as it is now live? :-)
  • [ON TRACK] Enable QA and contributors to increase their skills and participation by crafting automated tests for B2G and desktop smoketests by providing examples, mentoring, and documentation

Check out Pivotal Tracker

Highlights

This is a place for people to put announcements or to describe activities that they'd like to highlight. It's not the responsibility of the MC to find things to put here.

  • [edmorley] Tree Status now live! This switch means that we are one step closing to ditching tinderbox & makes the sheriff's lives much easier, since multiple trees can have their state changed simultaneously (eg for an infra-caused tree closure), status messages can now be saved/restored, changes are logged (so you know who to ping for questions) & we have a proper API!

In-Depth Project Discussion

Pick one project/presenter per week who will give a detail presentation on their project.

The Overview

Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!

AutoPhone/Noah's Ark

Bughunter

Bugzilla

Eideticker

  • [wlach] Finishing up dashboard support for Galaxy Nexus
  • Integrated some great patches from Benoit Girard (from the graphics team) - zooming around results is now much easier, get a summary of datapoints on hover. More exciting stuff pending.

Firebug Testing

Jetperf

Marionette

  • [mihneadb] almost done on getting Marionette to use Moztest for Autolog posting

Mobile Automation

  • [ahal] Investigating reftest resolution problem
  • [ahal] Made progress triaging, filing and fixing reftests for b2g
    • Reftests don't draw to canvas properly when running out of process
    • Some tests fail when using the <iframe> browser api instead of the xul:browser one
    • See bug 773482
  • [mihneadb] started work on getting the new agent to pass the existing tests.

Mobile Evangelism Site-Scraping Tool

Mochitest

MozBase

Mozmill

MozTrap

New Graphserver (Datazilla)

  • [jeads] The metrics team developed an exponential smoothing treatment for t-test results that is very effective in reducing noise in talos data. Working on integrating that into datazilla now.

Open Web Apps

Peptest

Powerball

Pulse

Signal From Noise

Speedtests

Systems

Talos

  • [jhammel] give xtalos a real API and use it
  • [jhammel] Talos is now sending production data to datazilla (with datazilla_client)

Sheriffing & Trees

  • [edmorley] Away today (27th) due to UK bank holiday.
  • [edmorley] Day-to-day sheriffing of the trees.
  • [edmorley] Sheriffing/tree closure issues of note:
  • [edmorley] Investigation & patch for bug 690672 - retriggering builds manually should never coalesce. This issue is a significant pain point for sheriffs on a daily basis.
  • [edmorley] Reviewed the revised list of removed 10.5 builders in bug 773120.
  • [edmorley] After the high infra load / long Try wait times issue was partially blamed on wasting capacity (by releng), filed bug 784681 & dependants to assess all hidden builds/tests on TBPL to (a) make sure bugs filed to fix or else switch off broken/potentially unwanted suites, and (b) ensure releng realise that most are in fact either working or blocked on releng themselves & so that they can't use them as a reason not to prioritise adding extra capacity.
  • [edmorley] Put together an idea for saving infra load, by only building certain platforms if files used by them changed. eg: If a push only changes b2g/* then don't build anything other than B2G! :-) (We already do something like this for spidermonkey builds triggered on changes to js/src/). Created a WIP for producing stats for how much load this would save - which when finished in the coming week will be posted to dev.platform and used as justification for why it would be a good use of releng's time (my gut feeling is that this will save us quite a bit).

Treestatus

Tinderboxpushlog

  • [edmorley] Gave feedback on buildbot patch part of bug 704006 (adding TBPL support for identifying self-serve cancelled jobs from those caused by infra issues).
  • [edmorley] Filed TBPL followups to TreeStatus support: https://bugzil.la/784670,785051,785058
  • [edmorley] Made further tweaks to README.vagrant in response to feedback from those unfamiliar with Vagrant (bug 764894).
  • [edmorley] Put together a list of easier, but still highly wanted TBPL bugs, so that a new contributor could pick some of them to work on.
  • [edmorley] Helped a contributor set up a local TBPL instance using Vagrant & debug networking issues due to a VirtualBox bug. Want to try and find the root cause, so we can make Vagrant work for everyone out of the box and not put off potential new TBPL contributors.
  • [edmorley] Discussions about bug 713419 (Use sheriffs.mozilla.org instead of the Google calendar to get the sheriff's name).
  • [edmorley] Testing of patch for bug 656155 (Add "go to build" link for non-Try builds), testing performance of alternatives & review comments.

War on Orange

  • [edmorley] OrangeFactor:
  • [edmorley] Intermittent orange work:
    • In run up to ESR17, putting a lot of effort into anti-orange work, since whatever is in mozilla17 we'll be stuck starring on TBPL until December 2013. Chased up more newly landed tests that are failing intermittently, CCed devs on more top, unowned oranges.
    • Tried to narrow the regression range for bug 784247.

Web QA

Automation Development

  • [henrik] Office hours upcoming this week on Thursday
  • [rwood] Submitted more tests for B2G WebTelephony API

Upcoming Events

Standing weekly meeting related to our projects should go here

Round Table

Put your questions and things to raise with the entire team here

Take notes here and copy and paste them to this section afterwards.

Notes

Holidays and Trips

Next week MC

Misc

Find something you think is cool, interesting, funny, or exciting and put it here

Retrieved from "https://wiki.mozilla.org/index.php?title=Auto-tools/Meetings/2012-08-27&oldid=464445"