Auto-tools/Meetings/2013-03-04
From MozillaWiki
< Auto-tools | Meetings
Contents
Goals Progress
Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.
- Goal: Continue to deliver B2G automation support for in house developers and partners - jgriffin & team
- Goal: Create new Performance measurement tool with performance team to measure startup using the best of our new tools - mozharness/mozbase/datazilla. - jmaher & perf team
- Goal: Support next generation android efforts by getting Android x86 virtualized environments to run mochitests - bc
- Goal: Improve test slave capacity by allowing linux desktop tests to be entirely virtualized for tests that do not need hardware support. - jmaher & Releng
- Goal: Build to support future capacity for workflow and performance by:
- Deploying Bugzilla 4.2 - dkl, glob
- We are going LIVE tomorrow. Upgrade starts at 1730 PST.
- Publicizing and starting on an effort to revamp TBPL - jeads, edmorley & team
- Updating test harnesses to use mozbase and mozharness and creating a migration plan for moving to better test reporting across the board (migration planned for Q2 to coincide with TBPL v2 delivery) - jhammel & ahal
- Revamp GoFaster measurement tools - camd
- Create plan for triage workflow and begin triaging according to it - lizzard
- Make all QA centered automation ready for rapid betas to allow the desktop team to work faster - Henrik & team
- Deploying Bugzilla 4.2 - dkl, glob
Highlights
This is a place for people to put announcements or to describe activities that they'd like to highlight. It's not the responsibility of the MC to find things to put here.
- OrangeFactor on trunk trees for last 7 days is now a whopping 10.35 !! :-( (See dev.platform thread).
Issues
Any issues that need discussing with the entire team go here
- New Meeting format
In-Depth Project Discussion
Pick one project/presenter per week who will give a detail presentation on their project.
Upcoming Events
- Bugzilla downtime & the long awaited upgrade is Tuesday night (i.e. tomorrow) in north america, 17:30PST - 11PST. If you know a bugzilla developer, send them snacks and coffee. But seriously, if any tools go down as a result, be prepared to be on point to fix them. However, we don't expect any issues.
Round Table
- Who can help out diving into the Android automation failures in the top 10 highest on orange factor?
- We need to land FPS measurements by end of week for b2g. We are close. What does it take to get there?
- There has been a recent surge in bug 710942 and bug 844648. Do we know what's causing these disconnects?
Take notes here and copy and paste them to this section afterwards.
The Overview
Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!
Mobile (1st Monday)
AutoPhone/Noah's Ark
- ran demo unittests for mozilla-inbound. Results available at http://mrcote.info/autolog_bc/?tree=mozilla-inbound&source=autolog
- ran demo s1s2 tests for 2012-10-01 to 2013-03-04 for all Virginia Haxxor phones. Results available at http://mrcote.info/phonedash_bc/
- Interesting events:
- Nov 8 - improvement in Local Blank stop
- Jan 2 - regression in Local Blank stop (related to telemetry pref change?)
- Jan 15-16 - big regression in Remote Blank page stop followed by big increase in volatility
- Interesting events:
- manually rerunning s1s2 tests for Virginia Haxxor from 2012-10-01 to 2013-03-04. Results available at http://mrcote.info/phonedash
- starting unittest reporting to brasstacks autolog for Galaxy GS2 and Nexus S. Results available at http://brasstacks.mozilla.com/autolog/?tree=mozilla-central&source=autolog
- It would be interesting to measure and report on the Read/Write performance of the sdcard and perhaps internal storage to get a feeling about whether a phone has degraded after too many writes.
B2G Automation
- Datazilla B2G Performance Dashboard: https://datazilla.mozilla.org/b2g/
- Gaia UI tests being run in TBPL per commit to Gaia: https://tbpl.mozilla.org/?tree=Gaia-Master
- B2G emulator tests other than reftests have been moved to Amazon VM's: https://tbpl.mozilla.org/?jobname=emulator
- [rwood] Developing long-running GaiaUI tests (bug 843083). Last week completed tests for airplane mode, SMS, video playback, browser, and edit/delete events. For a complete list please see the GaiaUI Stress Test etherpad.
Mobile Evangelism Site-Scraping Tool
Android Automation
- [wlach] mozdevice documentation nearly ready to go in: bug 845506
Tools (2nd Monday)
Bugzilla
- Upgrade to bmo/4.2 happens this week!
Bughunter
GoFaster
Pulse
MozTrap
Tinderboxpushlog
- [edmorley] New things in production: https://bugzil.la/841786,843524,844073,844102,844811
- [edmorley] Finishing up the TBPL Chief self-deploy coordination with IT (bug 827473).
- [edmorley] Reviewed multiple iterations of bug 841316.
War on Orange
- OrangeFactor on trunk for last 7 days: 10.35 !! :-( (See dev.platform thread)
- [edmorley] Reviewed bug 766546, bug 634747.
- [RyanVM] Regression finding & backouts of recent oranges.
- [RyanVM] Developer outreach on some of the unowned top oranges (bug 749894, bug 833769, bug 754860, bug 689256)
- [RyanVM, edmorley] Discussions with Clint about bringing OrangeFactor up in tomorrow's platform meeting and possibly offering once or twice monthly updates on an ongoing basis.
TBPL Version 2
- [edmorley] Created the treeherder-service github.com repo.
- [mdoglio] Pull request for initial Vagrant dev environment.
- [jeads] Implemented new treeherder (TBPL2) schema
Performance (3rd Monday)
New Graphserver (Datazilla)
- [jeads] Released a few bug fixes and features for https://datazilla.mozilla.org/b2g
- Adjusted database query and the graph displaying replicates in the UI to address the issue described in bug 845864
- Added "7 Days" to the list of time range filters to address bug 845364
- Application names that do not have data available for the selected test and branch are automatically hidden. This addresses bug 844756
- Added a select/unselect all checkbox in the top of the Applications container. Thought this would be useful as we approach double digit application lists.
- Added a permalink feature that captures the UI state and provides a handful of filter params in the url. This addresses bug 845936. Will be replacing this with a url based history this week, see bug 847174 for details
Signal From Noise
Speedtests
Eideticker
- [wlach] - used eideticker's startup test to evaluate pgo builds (found no difference, as with talos)
- [wlach] - evaluation continues for pointgrey cameras. have capture working with flea3 test model, though only in black & white for some reason
Talos
General Framework Support (4th Monday)
MozBase
WebRTC Test Development
Process (4th Monday)
Sheriffing & Trees
- [edmorley] WIP policy draft (destined for wiki.m.o) on "criteria for having a new platform/testsuite shown by default on TBPL". (In response to the same questions having arisen multiple times over the last few months).
- [edmorley] Several discussions over the tree.m.o aliases and tinderbox depreciation.
- [edmorley, RyanVM] Multiple debates over the need to close the trees for the Bugzilla 4.2 rollouts. The resistance received (and having to restate our case to multiple parties, when the sheriffs IMO ultimately have the final say) was pretty frustrating.
- [RyanVM] Day to day sheriffing of trees, including multiple tree closures for various infrastructure problems and bustage landing during peak checkin periods. As noted above, last week's OrangeFactor made regression hunting and backouts extremely difficult.
- [RyanVM] Initial triage and filing of mochitest-assertion-as-failures bugs.
- [RyanVM] Daily release branch uplifts. Weighed in on B2G developer request to consider tracking+ sufficient for uplift approval through 3/15.
Treestatus
Bugmasters
Holidays and Trips
- [edmorley] PTO next week Mon+Tues (11th-12th March).
- [jgriffin] in transit to and from Open Source Days in Copenhagen March 7,8 and 11, 12
Misc
Find something you think is cool, interesting, funny, or exciting and put it here