Auto-tools/Meetings/2012-11-19

From MozillaWiki
Jump to: navigation, search

Goals Progress

Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.

  • Ensure B2G has all test automation available to prepare for launch
    • [ON TRACK] Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:
      • [DONE] Mochitest-plain (emulator first, panda second)
      • [SKIPPED] Mochitest-chome (emulator first, panda second)
      • [DONE] Reftest (emulator first, panda second
      • [ON TRACK] xpcshell (panda)
      • [DONE] webAPI (emulator only
      • [ON TRACK] webQA - gaia front end tests emulator and panda
      • [ON TRACK] gaia dev integration/unit tests emulator and panda
    • [ON TRACK] Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng
    • [ON TRACK] Ensure all webAPIs testable through the emulator have comprehensive automated tests
    • [ON TRACK] Continue to aid QA in creating automated tests for B2G
  • Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.
    • [ON TRACK] Have APC.io (ARMv6)boards smoketest ready - rooted, working, plugged into Autophone
    • [ON TRACK] Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations
    • [ON TRACK] Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing
      • Should be done first version early this week.
    • [ON TRACK] Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.
  • Create and deploy performance automation for desktop, fennec, and B2G product benchmarks
    • [ON TRACK] Create Eideticker tests for B2G on panda boards and run them once a day
    • [ON TRACK] Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis
    • [ON TRACK] Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions
  • Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.
    • [ON TRACK] Deploy 4.2 version of Bugzilla
      • Waiting on IT...
    • [ON TRACK] Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.
    • [ON TRACK] Complete the 'Sandstone' CSS update
    • [ON TRACK] Rework browserID <--> BMO integration
    • [ON TRACK] Deliver user and product dashboards
    • [ON TRACK] Deploy tracking flags extension database optimization
  • Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem
    • [ON TRACK] Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL
    • [ON TRACK] Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures
      • [edmorley] Created v2 of the bookmarklet used to simplify the TBPL -> 'viewing job on the buildbot master' workflow (bug 809908).
      • [edmorley] Fixing the test harness issues on Android that cause us to miss all crashes on robocop and all shutdown crashes on mochitest/crashtest/reftest/jsreftest (bug 808410). Also fixed similar issues with checking for hung processes (bug 812135).
      • [edmorley] Parser improvements: https://bugzil.la/812205,812207,812214
      • [edmorley] Adding an easier way to look up the slave history for the machine that performed a job (bug 736085).
    • [DONE] Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.
    • [ON TRACK] Make orange factor update more than once a day to help make it easier to see changes over time
    • [ON TRACK] Assist with getting Datazilla's UI integrated with TBPL, phase 1
  • Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.
    • [ON TRACK] Re-organize and improve the automated testing documentation at MDN
    • [ON TRACK] Develop first round of unit tests for WebRTC
    • [ON TRACK] Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage

Highlights

This is a place for people to put announcements or to describe activities that they'd like to highlight. It's not the responsibility of the MC to find things to put here.

In-Depth Project Discussion

Pick one project/presenter per week who will give a detail presentation on their project.

The Overview

Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!

AutoPhone/Noah's Ark

Bughunter

Bugzilla

  • http://globau.wordpress.com/2012/11/15/268/
  • [glob] significant performance related investigations, resulting in much discussion and a few patches
  • [glob] bugzilla push extension deployed to production
  • [dkl] significant progress on the mozilla skin for bmo
  • [glob+dkl] lots of upstream patches and reviews

Eideticker

Firebug Testing

Jetperf

Marionette

Mobile Automation

  • [mcote] mozpool back end just about complete (~1-2 days).
  • [ahal] wasted most of the week on bug 809437
    • working on getting b2g tests unhidden on tbpl
    • also on getting unittests to dump logcat so we can debug these issues more easily in the future

Mobile Evangelism Site-Scraping Tool

Mochitest

MozBase

Mozmill

MozTrap

  • [camd] - Still working on MozTrap perf issues. My fixes help, but may need more attention.
  • [camd] - In talks with Clint / Mark Cote / Matt Brandt about continuing working on MT into January and what features need completion prior to mid-December.

New Graphserver (Datazilla)

Open Web Apps

Peptest

Powerball

Pulse

Signal From Noise

Speedtests

Systems

Talos

Sheriffing & Trees

  • [edmorley] Day-to-day sheriffing of the trees.
  • [edmorley] Collating the various new Panda/B2G failure modes, ensuring all filed, hiding emulator tests due to failure rate. Will hopefully soon be able to be unhidden (bug 812149 and dependants).
  • [edmorley] Created v2 of the bookmarklet used to simplify the TBPL -> 'viewing job on the buildbot master' workflow (bug 809908).
  • [edmorley] Patches:

Treestatus

Tinderboxpushlog

War on Orange

  • [edmorley] Another big attack on our OrangeFactor this week. Last 7 days are at 1.35 :-D
    • Though OF currently doesn't recognise B2G/emulator failures (bug 812551) & our test harnesses miss some android crashes (bug 808410) - so once those are fixed we'll see a slight increase again. (They aren't new issues - we've just not been counting them until now).

Web QA

Automation Development

  • [rwood] WebAPI emulator tests for Proximity Sensor

Upcoming Events

Standing weekly meeting related to our projects should go here

Round Table

  • B2G Deadline December 10.


Take notes here and copy and paste them to this section afterwards.

Notes

Holidays and Trips

  • US out Thursday and Friday this week

Next week MC

Misc

  • PyCon Canada had some interesting talks!
    • Science and Python: Retrospective of a (mostly) successful decade (video, slides (seems to be a simpler version of his actual PyCon.Ca talk))
      • This was amazing. He got a standing ovation. :)
    • Twisted History of Python Packaging (video, slides)
    • Python for Humans (by the author of the Requests package) (video)
    • How you can contribute to Python (video)
    • Beyond Passwords: Secure Authentication with Mozilla Persona :) (video)
    • Wilderness Labs (video)
    • Network Programming for Automation Developers :) (video, slides)
  • Mobile Test Summit talks are up!
Retrieved from "https://wiki.mozilla.org/index.php?title=Auto-tools/Meetings/2012-11-19&oldid=488713"