EngineeringProductivity/Meetings/2011-09-26

RoundTable

Let's keep notes over here as we do the discussion.

• Discussion on projects, priority and polish
  • Two phase solution - development & release
  • Having well-defined priorities
  • Having well-defined end goals for each phase of a project
  • Handling bumping projects and shifting priorities

• Outcome: What can we do (as a team) to help us with this?
• Outcome: What can Clint do (as team lead) to help with this?

Notes from the etherpad

TODO:
* get a better videoconferencing system for MV
** +1
** +1000
decisions require information
would clearer definitions of projects help? clearer tracking?
what if we write a better set of documentation before starting the project?  either wiki/blog
phase 1 (prototype) hard to estimate, since it's largely experimentation
 * nightly - smaller prototypes
phase 2 (release) easier to estimate, since we have a good idea of what's to be done
 - more detailed plans & time estimates
 * aurora
 phase 3
 * beta
(related to above) Use more ideas from SCRUM (e.g. sprints) ? : 
http://en.wikipedia.org/wiki/Scrum_%28development%29
a-team has history of taking projects and being able to deliver, and now the number of projects coming in is growing. How to manage this? How to determine which project should take priority over another?
We have older projects that don't have owners, or have a suffient set of features implemented. Need to assign owners or pass the projects off to the right group.
Sometimes priorities in the company change and projects which were previously high priority are no longer so important (e.g. firebug automation). Should we be more explicit when this happens and we stop working (as much) on certain projects?
we should all be inserted into elastic search!
Customer management: feedback from customers
* customer might be us!
what are other teams doing to track multiple different-sized projects?

Goals: These are the goals I pulled out of the "Thinking Caps" thread, plus some that I know about. The details are not necessarily complete, but listing them all helps illuminate the big picture of where we're heading.

• mobile automation
  • get from 25% failure rate to <5% rate (desktop parity that is)
  • make it easier for developers to run and debug tests with tools and hardware
    • Make targets for all tests
    • better documentation for all tests and tools
  • Investigate pandaboards for honeycomb support?

• performance
  • aid in future xperf and pageloader modifications
  • keep talos and AMO perf up to date with firefox changes (i.e. missing application.ini, etc..)
  • resolve and triage perf bugs

• specialpowers
  • finish the mochitest harness
  • ensure existing work done on tests lands (i.e. content patches from :jdm)
  • make a specialpowers-core for quit, prefs and logging. Good for talos, etc...
  • NOTE: this will make us 100% harness and about 75% test complete.

• mozbase
  • have working prototype with releng for talos, mochitest, reftest, xpcshell
  • resolve location and process for mozprocess, mozprofile, etc...
  • land any prerequisite patches and toolchains
  • NOTE: I suspect we won't get mozbase landed 100%, but we should be getting close
  • get virtualenv on m-c
  • coordiante with Release Engineering on mobase/mozharness efforts
  • get talos on mozbase packages
  • get automation.py on mozbase packages
  • update documentation
  • communicate about this!
  • continue to added needed features to mozbase packages
  • bugs are at https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=mozbase&list_id=1352142
  • [MAYBE] split up the mozmill repo
• manifests
  • mochitest-plain manifests

• MozMill:
  • get 2.0 out the door for real and support it
  • possibly help refactor mozmill-automation if desired
  • Make Mozmill e10s ready (probably working with :ahal)

• W3C browser testing work
  • TBD

• B2G
  • Get phase 1 of platform level test framework created
  • Aid with build system needs (joint with releng?)
  • Have linux agent ready to go

• Tegra Automation
  • Deploy to MV network and announce
  • Wire up with panda boards as they come available?

• SpeedTests
  • Expanding to new browsers
  • Expanding tests
  • Maintenance

• Eideticker
  • Complete test framework side
  • Investigate using another card?

• WOO
  • Shepherd through secreview

• Templeton
  • Find/fix

• bughunter
  • Greater integration with soccoro?

• User Responsiveness Regression Test
  • Complete harness for this
  • Shepherd through buildbot and/or mozharness integration

• Better community outreach
  • Have volunteers involved in community stewards effort
  • Have a blogging goal for the team?

• Better process/priority clarity for each project
  • TBD (see above)

Goals

• Work with release engineering to reduce overall end-to-end time from push to test result to an average of 2 hours.
• Ensure all harnesses and tests are E10S ready
• Improve performance testing by finalizing user-activated addons performance system, xperf-enabled performance testing, and automating a user responsiveness performance test.

Automation

• Go Faster (P1) - jgriffin, ted, mdas, bc, ctalbert, wlach
  • Some minor further improvements pushed to gofaster dashboard (see: bug 688588, bug 686034)
  • Probably easing away from further improvements to dashboard for now. Some bugs still unfixed (e.g. bug 686036, bug 687600). Thoughts?

• Mobile Support (P1) - jmaher, bmoss, ctalbert

• Bug Hunter (P1) - bc

• Mozmill 2.0/1.5.x (P1) - jhammel, harth, ctalbert
  • two bugs left!
  • will release a new beta today
  • other than that, we're in a holding pattern

• Crossweave 2 - jgriffin

Performance

• Talos/User Responsiveness (P1) - anode, ted

• Talos/Addon Testing Service (P1) - anode

• Talos/General (P1) - anode

• Talos/Xperf (P2) - jmaher

• SpeedTests (P2) - mcote
  • New results front end: http://brasstacks.mozilla.com/speedtests/results.html
  • Updating docs so that ahal can take over machine maintenance while I'm away.

• Eideticker (P2) - mcote, wlach
  • Project synopsis now on public wiki
  • Video capture prototype now up as a github project
  • Discussion of architecture issues, etc. last week
  • Going with talos for now as a harness
    • First iteration will just capture loading static pages
    • Second iteration may use more complex simulated user interactions. Some initial thoughts in this message.
  • Video capture machine (+ LG Android device with HDMI out) now at wlach's house

Web Tools

• War on Orange/Auto-log (P1) - jgriffin, mcote
  • View for Bug Correlations awaiting review.

• Bugzilla Improvements (P1) - dkl, glob
  • The decision has been made to modify the core Bugzilla code where appropriate to best suit how Mozilla uses it; this should result in faster improvements to Bugzilla with a tighter focus on our requirements
  • Tweaks to searching (comments not searched by default, google searching integrated)
  • Various bug fixes and tweaks

• Regression Hunter (P1) - harth

• DXR Automation/Support {P2} - mdas
  • Submitted automation code to jcranmer (works on DXR). When he hooks up that code, along with bug 688673 landing, then build reports should be coming into autolog on a daily basis.

• Personal Bugzilla Dashboard (P2) - harth

Tool Infrastructure

• SpecialPowers (P1) - jmaher, ted

• MozBase (P2) - jmaher, jhammel, ted

• Web UX Platform (P2) - mcote, harth
  • A couple more useful JS libraries added in.

• flyingtanks - mcote
  • WOO update complete, no security concerns.
    • Will be installing it on flyingtanks soon (but brasstacks' copy will be up until everything is reviewed & migrated).