Auto-tools/Meetings/2012-11-05
From MozillaWiki
< Auto-tools | Meetings
Contents
- 1 Goals Progress
- 2 Highlights
- 3 In-Depth Project Discussion
- 4 The Overview
- 4.1 AutoPhone/Noah's Ark
- 4.2 Bughunter
- 4.3 Bugzilla
- 4.4 Eideticker
- 4.5 Firebug Testing
- 4.6 Jetperf
- 4.7 Marionette
- 4.8 Mobile Automation
- 4.9 Mobile Evangelism Site-Scraping Tool
- 4.10 Mochitest
- 4.11 MozBase
- 4.12 Mozmill
- 4.13 MozTrap
- 4.14 New Graphserver (Datazilla)
- 4.15 Open Web Apps
- 4.16 Peptest
- 4.17 Powerball
- 4.18 Pulse
- 4.19 Signal From Noise
- 4.20 Speedtests
- 4.21 Systems
- 4.22 Talos
- 4.23 Sheriffing & Trees
- 4.24 Treestatus
- 4.25 Tinderboxpushlog
- 4.26 War on Orange
- 4.27 Web QA
- 4.28 Automation Development
- 5 Upcoming Events
- 6 Round Table
- 7 Misc
Goals Progress
Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.
- Ensure B2G has all test automation available to prepare for launch
- [ON TRACK] Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms.
- [ON TRACK] Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng
- [ON TRACK] Ensure all webAPIs testable through the emulator have comprehensive automated tests
- [ON TRACK] Continue to aid QA in creating automated tests for B2G
- Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.
- [ON TRACK] Have APC.io boards chassis-ready -- rooted, smoketested, and working for ARMv6 testing
- [ON TRACK] Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations
- [ON TRACK] Implement a new pool-based approach for device management in automation to increase stability on all device testing
- [mcote] Put together a solid MozPool design integrated into dustin & ted's BMM work. Coding about 50% done.
- [ON TRACK] Deploy Autophone with 40 phones and 70% up time
- Create and deploy performance automation for desktop, fennec, and B2G product benchmarks
- [ON TRACK] Create Eideticker tests for B2G on panda boards and run them once a day
- [ON TRACK] Port Eideticker to win32 and run tests there once a day to measure win32 snappiness for desktop
- [ON TRACK] Create pageload benchmark with networking shaper tests so that we can adequately track user-perceived performance across all devices and network types
- [ON TRACK] Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis
- [ON TRACK] Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions
- Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.
- [ON TRACK] Complete the 'Sandstone' CSS update
- [ON TRACK] Complete the REST API integration
- [ON TRACK] Rework browserID <--> BMO integration
- [ON TRACK] Deliver user and product dashboards
- [ON TRACK] Deploy tracking flags extension database optimization
- Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem
- [ON TRACK] Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL
- [edmorley] Unagi builds and marionette tests are now unhidden on all trees :-D
- [ON TRACK] Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures
- [edmorley] If a filename can't be found in a log failure line, TBPL now searches for bug suggestions using the whole line - meaning we can now star 20+ more types of failures \o/ (bug 807707).
- [edmorley] Test harness patches to make more failures on TBPL starable / easier to diagnose: https://bugzil.la/757838,808419,808410,808417
- [DONE] Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.
- [ON TRACK] Make orange factor update more than once a day to help make it easier to see changes over time
- [ON TRACK] Assist with getting Datazilla's UI integrated with TBPL, phase 1
- [ON TRACK] Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL
- Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.
- [ON TRACK] Finish and deploy mozmill 2.0 changes to frameworks and existing QA tests
- [ON TRACK] Update selenium grid to use ESX on-demand to improve web QA capacity
- [ON TRACK] Re-organize and improve the automated testing documentation at MDN
- [ON TRACK] Develop tests for WebRTC, B2G and other high profile projects
Highlights
This is a place for people to put announcements or to describe activities that they'd like to highlight. It's not the responsibility of the MC to find things to put here.
- [edmorley] TBPL now supports starring ~20+ more of the common intermittent failures \o/ (bug 807707 & bug 757838).
In-Depth Project Discussion
Pick one project/presenter per week who will give a detail presentation on their project.
The Overview
Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!
AutoPhone/Noah's Ark
- [mcote] Put together a solid MozPool design integrated into dustin & ted's BMM work. Coding about 50% done.
Bughunter
Bugzilla
- [glob] Out sick
- [dkl+glob] More upstream reviews in prep for security release and upcoming version 4.4
- [dkl] Script work for Orangefactor keyword migration
- [dkl+glob] Normal administrative work
Eideticker
Firebug Testing
Jetperf
Marionette
Mobile Automation
Mobile Evangelism Site-Scraping Tool
Mochitest
MozBase
Mozmill
MozTrap
- [camd] - Run Tests now displays results from other users for the same case / env so you don't repeat work. Don't have it auto-updating with ajax yet.
- [camd] - should deploy run series this week.
New Graphserver (Datazilla)
Open Web Apps
Peptest
Powerball
Pulse
Signal From Noise
Speedtests
Systems
Talos
Sheriffing & Trees
- [edmorley] Monday: travel day.
- [edmorley] Day-to-day sheriffing of the trees.
- [edmorley] Bisected "Windows-PGO-no-profiling-only bustage" for Ehsan (bug 805862). Boo to 4 hour PGO turnaround times :-(
- [edmorley] Discussed using a transparent HTTP cache for BzAPI to reduce b.m.o load (bug 805763 & bug 805109).
- [edmorley] Drove forwards several bugs relating to switching from [orange] to the keyword 'intermittent-failure' (bug 790571 and dependants).
- [edmorley] Requested the addition of TBPL, treestatus & MXR to status.mozilla.com.
- [edmorley] Unagi builds and marionette tests are now unhidden on all trees :-D
- [edmorley] Patches:
- Test harness patches to make more failures on TBPL starable / easier to diagnose: https://bugzil.la/757838,808419,808410,808417
- Other test harness patches: https://bugzil.la/806995,803413
Treestatus
Tinderboxpushlog
- [edmorley] New things in production: https://bugzil.la/799578,805954,806669,807036,807707
- [edmorley] Patches:
- If a filename can't be found in a log failure line, TBPL now searches for bug suggestions using the whole line - meaning we can now star 20+ more types of failures \o/ (bug 807707).
- Other patches: https://bugzil.la/807120,807036
- [edmorley] Initiated the move of TBPL from a user repo to to hg.m.o/webtools/tbpl/ (bug 772546 & bug 807008).
- [edmorley] More work to get the in-production, tbpl-dev & in-tree TBPL DB schemas synced up and optimised (bug 799578 & bug 806121).
- [edmorley] Reviewed bug 806669 and bug 808017.
- [edmorley] Helping a contributor with more of bug 683833.
War on Orange
- [edmorley] Patches for OrangeFactor & bzcache for the switch to using the keyword 'intermittent-failure': https://bugzil.la/807113,807114
- [edmorley] Ongoing backouts of newly introduced flaky tests, disabling chronic top-oranges & backporting orange fixes to aurora/beta/esr.
Web QA
Automation Development
Upcoming Events
Standing weekly meeting related to our projects should go here
Round Table
Put your questions and things to raise with the entire team here
Take notes here and copy and paste them to this section afterwards.
Notes
Holidays and Trips
- [mcote] at PyCon Canada this coming weekend. Travelling Nov. 9 and 12 on the train, so will have (limited) Internet connectivity.
- [edmorley] PTO on Friday 9th.
Next week MC
Misc
Find something you think is cool, interesting, funny, or exciting and put it here