Auto-tools/Meetings/2012-11-26
From MozillaWiki
< Auto-tools | Meetings
Contents
- 1 Goals Progress
- 2 Highlights
- 3 In-Depth Project Discussion
- 4 The Overview
- 4.1 AutoPhone/Noah's Ark
- 4.2 Bughunter
- 4.3 Bugzilla
- 4.4 Eideticker
- 4.5 Firebug Testing
- 4.6 Jetperf
- 4.7 Marionette
- 4.8 Mobile Automation
- 4.9 Mobile Evangelism Site-Scraping Tool
- 4.10 Mochitest
- 4.11 MozBase
- 4.12 Mozmill
- 4.13 MozTrap
- 4.14 New Graphserver (Datazilla)
- 4.15 Open Web Apps
- 4.16 Peptest
- 4.17 Powerball
- 4.18 Pulse
- 4.19 Signal From Noise
- 4.20 Speedtests
- 4.21 Systems
- 4.22 Talos
- 4.23 Sheriffing & Trees
- 4.24 Treestatus
- 4.25 Tinderboxpushlog
- 4.26 War on Orange
- 4.27 Web QA
- 4.28 Automation Development
- 5 Upcoming Events
- 6 Round Table
- 7 Misc
Goals Progress
Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.
- Ensure B2G has all test automation available to prepare for launch
- [ON TRACK] Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:
- [DONE] Mochitest-plain (emulator first, panda second)
- [SKIPPED] Mochitest-chome (emulator first, panda second)
- [DONE] Reftest (emulator first, panda second
- [ON TRACK] xpcshell (panda)
- [DONE] webAPI (emulator only
- [ON TRACK] webQA - gaia front end tests emulator and panda
- [ON TRACK] gaia dev integration/unit tests emulator and panda
- [ON TRACK] Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng
- [ON TRACK] Ensure all webAPIs testable through the emulator have comprehensive automated tests
- [ON TRACK] Continue to aid QA in creating automated tests for B2G
- [ON TRACK] Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:
- Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.
- [ON TRACK] Have APC.io (ARMv6)boards smoketest ready - rooted, working, plugged into Autophone
- [ON TRACK] Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations
- [ON TRACK] Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing
- Back end done and stable with exception of SUT lifeguard, waiting on bootable B2G image.
- Request UI done by tomorrow.
- [ON TRACK] Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.
- Create and deploy performance automation for desktop, fennec, and B2G product benchmarks
- [ON TRACK] Create Eideticker tests for B2G on panda boards and run them once a day
- [ON TRACK] Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis
- [ON TRACK] Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions
- Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.
- [ON TRACK] Deploy 4.2 version of Bugzilla
- Tentatively scheduled for night of 9 December. Slightly later than planned (due to delays in getting new clusters set up) but still on track.
- [ON TRACK] Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.
- [ON TRACK] Complete the 'Sandstone' CSS update
- [DEFER] Rework browserID <--> BMO integration
- Deferred in favour of some optimizations, github-migration script, and various one-off requests.
- [ON TRACK] Deliver user and product dashboards
- [DEFER] Deploy tracking flags extension database optimization
- Needs to happen after 4.2, but that goes into holidays. Deferring to early 2013Q1.
- [ON TRACK] Deploy 4.2 version of Bugzilla
- Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem
- [ON TRACK] Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL
- Everything open has been completed, but not marking as DONE yet, since not sure what else may be requested.
- [ON TRACK] Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures
- See Sheriffing & Trees, Tinderboxpushlog & War on Orange sections.
- [DONE] Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.
- [ON TRACK] Make orange factor update more than once a day to help make it easier to see changes over time
- [ON TRACK] Assist with getting Datazilla's UI integrated with TBPL, phase 1
- Waiting for Datazilla completion.
- [ON TRACK] Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL
- Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.
- [ON TRACK] Re-organize and improve the automated testing documentation at MDN
- [ON TRACK] Develop first round of unit tests for WebRTC
- [ON TRACK] Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage
Highlights
This is a place for people to put announcements or to describe activities that they'd like to highlight. It's not the responsibility of the MC to find things to put here.
- [edmorley] We've now switched from marking intermittent failure bugs with whiteboard '[orange]' to a new keyword 'intermittent-failure' (bug 790571). The alias:randomorange bug has also been depreciated. Please file new bugs with the keyword to ensure TBPL, OrangeFactor & other tools see them.
In-Depth Project Discussion
Pick one project/presenter per week who will give a detail presentation on their project.
The Overview
Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!
AutoPhone/Noah's Ark
Bughunter
Bugzilla
- Off-schedule BMO push - http://globau.wordpress.com/2012/11/20/happy-bmo-push-day-20
- [glob] performance related work with better caching of certain Bugzilla objects
- [glob] Advanced search fixes committed upstream and for bmo-4.2
- [dkl] More progress on the mozilla skin for bmo and dashboards
- [dkl] Orange factor work with migration from whiteboard to keyword
- [glob+dkl] Needinfo extension fixes
- [glob+dkl] upstream patches and reviews, and BMO administrative tasks
Eideticker
Firebug Testing
Jetperf
Marionette
Mobile Automation
- [ahal] modified xpcshell harness so it doesn't depend on having an objdir available, bug 797153
- [ahal] started work to make mochitests support permissions testing
- [mcote] mozpool back end & API stable and deployed. Requests UI done by tomorrow.
Mobile Evangelism Site-Scraping Tool
Mochitest
MozBase
Mozmill
MozTrap
New Graphserver (Datazilla)
Open Web Apps
Peptest
Powerball
Pulse
Signal From Noise
Speedtests
Systems
Talos
Sheriffing & Trees
- [edmorley] Day-to-day sheriffing of the trees.
- [edmorley] Finished the switch from whiteboard '[orange]' to keyword 'intermittent-failure' (bug 790571 and dependants).
- [edmorley] Tracked down some of our most chronically unreliably tegras and filed bugs to disable them.
- [edmorley] Unhid B2G emulator tests now they are more reliable & the failure output more useful (bug 812149 and dependants).
- [edmorley] Hid Win64 on all trees for bug 814009.
- [edmorley] Finished up & landed the patch to stop us missing many Android crashes (bug 808410).
- [edmorley] Worked with IT to track down buildbot ftp.m.o timeouts (bug 811063).
- [edmorley] Patches:
- automationutils.py's checkForCrashes() should output the top frame of a crash in a TBPL-parsable format (bug 813650).
- Switching off of SunSpider in Talos, in preference to AWFY (bug 814270).
- Robocop work: https://bugzil.la/814011,814013
- Misc: https://bugzil.la/813025,814116,812513,793782
Treestatus
Tinderboxpushlog
- [edmorley] New things in production: https://bugzil.la/812103,812205,812207,812214,812472,736085,807120,812558
- [edmorley] Patch to improve parser support: bug 813039.
War on Orange
- [edmorley] OrangeFactor cleanup now that we're no longer using [orange] for intermittent bugs (bug 814394).
- [edmorley] Ongoing backouts of newly introduced flaky tests, disabling chronic top-oranges & backporting orange fixes to aurora/beta/esr.
Web QA
Automation Development
- [rwood] Developed two WebAPI tests for Proximity Sensor API (under review); raised issue concerning two of the event attributes bug 814043
- [rwood] Developed two WebAPI tests for verifying marking SMS messages read/unread (under review)
- [rwood] Developed a WebAPI WebTelephony test to verify correct behaviour when attempt to have more than one call on hold simultaneously
- [rwood] Began developing a WebAPI test to verify ability to send/receive multipart SMS with maximum segments (10); raised issue as simulating incoming SMS of 3 or more segments fails on 2nd try bug 814761
Upcoming Events
Standing weekly meeting related to our projects should go here
Round Table
- Presentations every week?
Take notes here and copy and paste them to this section afterwards.
Notes
- wlach suggested having a presentation every week to add some value to having all of us get together.
- Every person on the team should do a presentation (with 20 people, that's about twice a year per person).
- We would maintain a spreadsheet with the last time each person did a presentation, along with a queue of presentations to come.
- A lot of prep time is unnecessary, 10 or 15 minutes should do, unless you want to do more, of course.
- Try to avoid relying on Vidyo if possible for the benefit of people dialled in. Provide screenshots (before the meeting) if necessary.
Holidays and Trips
Next week MC
Misc
Find something you think is cool, interesting, funny, or exciting and put it here