Auto-tools/Meetings/2012-09-10

From MozillaWiki
Jump to: navigation, search

Goals Progress

Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.

  • [ON TRACK] Support the B2G and platform team by moving existing automation to that platform as well as expanding automation on B2G and B2G devices for basecamp deliverables.
    • [jgriffin] Setup gecko update server for testing B2G over-the-air gecko updates.
  • [ON TRACK] Expand Mobile testing platforms for B2G and Fennec Native to include user devices and NEON Chipsets (panda boards).
  • [ON TRACK] Establish new Datazilla system as a useful tool for capturing and diagnosing talos performance regressions on a per-push basis, as well as capturing data for network and B2G performance tests.
    • NOTE: This is largely a change to the data harvesting and analysis side of the Talos system (even though some changes are required to the test harness and tests themselves). We will not be replacing existing Talos data harvesting system in Q3. By the end of Q3, we want to have the new performance data harvesting and analysis system Datazilla to a point at which it can shadow the old system and we can use Q4 to verify that the new approach captures just as many regressions as the old system does. Potential timeframe for conversion of the old system to the new Datazilla system is in Q1 2013 (earliest "thumb in the wind" estimate).
    • [jeads & camd] - Landed the metrics and stats branch in the datazilla repository. This includes welch's t test, false discovery rate (fdr), and an exponential smoothing algorithm that each consecutive push to mercurial is run through. Working on pushing new repo to production this week.
  • [ON TRACK] Upgrade BMO to 4.2, deploy tracking flags optimizations, UX improvements (comment tagging, mid-air resolution)
  • [ON TRACK] Make TBPL More Orange aware to make it easier to file intermittent orange bugs, and so that we can more adequately track (and therefore address) issues around android-related and memory leak intermittent failures
  • [ON TRACK] Enable QA and contributors to increase their skills and participation by crafting automated tests for B2G and desktop smoketests by providing examples, mentoring, and documentation

Check out Pivotal Tracker

Highlights

This is a place for people to put announcements or to describe activities that they'd like to highlight. It's not the responsibility of the MC to find things to put here.

  • [wlach] / [BenWa] Preliminary Eideticker integration with the SPS Profiler has landed and it is awesome! This basically allows you to correlate a particular result on the eideticker dashboard with a profile of what Firefox/Fennec happened to be doing at a particular time. Here's an example (note that there is not yet a direct correspondence between the video and the profile, but that's coming soon)
  • MozCampEU in Warsaw (Rob and Henrik)
    • Our 'Make sure your code works' session was a success even that we run into a couple of issues; but participants are eager to continue in setting up the environment to run tests at home and hopefully coming back to us for active contribution
    • [henrik] Talked to >10 participants about test automation in general
      • Most likely a bunch of new upcoming contributors for l10n tests (Arky, Anas)
      • Sergej (Ukrain) who is working on Selenium tests will get more experience in Mozmill - interested in joined projects together with his colleagures
      • Advertised our meeting ans Q&A session to everyone
  • [edmorley] Very close to having TBPL prefetching of failure summaries server-side, for much more responsive UX in the client - saving sheriffs significant amounts of time per day. Swatinem & myself have debugged the last of the issues holding bug 718632 back - is now working on tbpl-dev (password ; bug 789495), almost ready for rollout to prod :-)

In-Depth Project Discussion

Pick one project/presenter per week who will give a detail presentation on their project.

The Overview

Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!

AutoPhone/Noah's Ark

Bughunter

Bugzilla

  • pushed http://globau.wordpress.com/2012/09/06/happy-bmo-push-day-12/
  • [dkl+glob] upstream development and reviews
  • [dkl] significant improvements to bmo landed (cc changes no longer trigger a mid-air, orange factor integration)
  • [dkl] drove upstream release, branch for 4.4 development
  • [glob] pushed along request for access to bmo's logs and database
  • [glob] ongoing work on bugzilla search and comment tagging

Eideticker

  • [mdas] patches undergoing code review and revisions, but need to add panda support to orangutan before all can land.
  • [wlach] per highlights/goals, galaxy nexus and sps profiler integration work

Firebug Testing

Jetperf

  • [jhammel] debugging jetperf on windows

Marionette

  • [mdas/jgriffin] making relevant changes to marionette so webQA team can start using it for Marketplace testing

Mobile Automation

  • [mdas] code review is beginning for a b2g helper class, which can be used for any testing framework that needs to manage b2g devices bug 789305
  • [mcote], [mihneadb] C++ SUT agent passes all relevant SUT tests (all except data channel (not yet implemented) and unzip (implemented in a different way)). Need to do some work on exec and that should make it ready for initial use by B2G team.
  • [mihneadb] XPCShelltest can now post to Autolog

Mobile Evangelism Site-Scraping Tool

  • [mihneadb]
    • hooked database to UI
    • UA-based content comparison
    • prefixed CSS property data processing

Mochitest

MozBase

  • [wlach] mozdevice now lives in testing/mozbase (finally!). See bug 723107.
  • [jhammel] mozbase mirrored to m-c
  • mozfile will soon exist; what is it? a place to put all file-related common-use python code that currently has no home
  • [mcote] Small fix to DeviceManagerSUT; added a unit test.
  • [mihneadb] improvements to Moztest for supporting Autolog posting by Marionette and XPCShelltest

Mozmill

MozTrap

  • [camd] Been talking to the B2G team about a few features they need
  • [camd] Began investigating how much work some of those features would be.

New Graphserver (Datazilla)

  • [jeads] - Completed metrics schema and algorithm implementation: this includes welch's t test, fdr, and exponential smoothing.
  • [jeads] - Finishing web service methods for metrics data retrieval this week.
  • [jeads] - Also, working on the production release this week. This will be relatively painful, both projects making use of datazilla have deposited corrupt data. This data will need to be deleted when we roll out the new repository, so there will be some complex database scripts that need to be executed.
  • [jeads] - Adding a new manage command that will be used by the stoneridge project to automatically combine recent branches on the landing page that displays the data charts.
  • [camd] - Completed the statistic webservice branch and jeads merged it.
  • [camd] - Worked on the new testdata webservice branch. Webservice for fetching raw test data is complete. Waiting on metrics schema branch code to implement webservice to fetch metrics data.

Open Web Apps

Peptest

Powerball

Pulse

Signal From Noise

Speedtests

Systems

Talos

Sheriffing & Trees

  • [edmorley] Day-to-day sheriffing of the trees.
  • [edmorley] Updated patch for bug 787019.
  • [edmorley] Landed bug 786290 & bug 690672 to disable coalescing when retriggering on TBPL.
  • [edmorley] Discussion about when we should email individual authors for talos regressions (bug 752002).
  • [edmorley] Posted to the dev.platform thread about the current state of talos testing.
  • [edmorley] Posted to the dev.planning thread about use of Try (or not) prior to landing on inbound.
  • [edmorley] Discussions about switching off spidermonkey builds vs unhiding in bug 783740.
  • [edmorley] Finishing up work on my pushlogparser which will be used both as a POC and to tweak settings for the "don't build on certain platforms if the diff was a no-op" idea. Will be posting about it this week.
  • [edmorley] Disucssions with mconner about his multiple mozilla-inbounds proposal. After consideration, I don't feel it gains us much at least until push races become intolerable (by which point we should really consider using AutoLand to deal with it instead).

Treestatus

Tinderboxpushlog

  • [edmorley] Diagnosing issues with Vagrant caused by the log prefetching patch (bug 718632).
  • [edmorley] Discussions about making it easier for new contributors to test on tbpl-dev (given that it is currently behind htaccess auth), filed bug 789495.
  • [edmorley] Debugging of bug 718632 (TBPL should prefetch logs), asking IT to fix dependencies (bug 788197 & bug 789506), testing once fixed.
  • [edmorley] Discussing suggested approach for bug 683833 with a new contributor.
  • [edmorley] Reviewed:
    • bug 656155 - Add "go to build" link for non-Try builds
    • bug 718632 - Fix daily import after prefetching patch landed + fix fclose warning.
    • bug 785051 - Add message of the day support to tree status
  • [edmorley] Patches:
    • bug 787922 - Vagrant: Display log output if errors occurred during provision
    • bug 787950 - Vagrant: Save output of import-buildbot-data.py to logs/import-log-<datetime>.txt
    • bug 788020 - Vagrant: Ensure import-buildbot-data.py waits until the required packages are installed & other cleanup
    • bug 733556 - getRevisionBuilds.php query should use 'builders.buildername IS NOT NULL' to be more efficient
    • bug 718632 - Make import-buildbot-data.py output the buildername and revision when prefetching logs
    • bug 718632 - Update bugscache schema to the revised version that landed on production
    • bug 788498 - Recognise talos 'chromez' as another form of 'Talos chrome'

War on Orange

  • OrangeFactor extension on Bugzilla has landed. Off by default; enable through Preferences. There's one fix (bug 789066) waiting to be deployed on Thursday.

Web QA

Automation Development

  • [henrik] About to finish the ESX project - only Windows 8 machines are left

Upcoming Events

Standing weekly meeting related to our projects should go here

Round Table

Take notes here and copy and paste them to this section afterwards.

Notes

Holidays and Trips

Next week MC

Misc

Find something you think is cool, interesting, funny, or exciting and put it here

Retrieved from "https://wiki.mozilla.org/index.php?title=Auto-tools/Meetings/2012-09-10&oldid=469182"