QA/StaffMeetings/QA-Staff-01-25-2012

From MozillaWiki
Jump to: navigation, search

« previous mtg | QA Staff Home | next mtg »

QA Staff Meeting Notes January 25, 2012

Meeting Details

 # Every other Weds @ 1:30p Pacific Time - Mozilla HQ, Warp Core 
 #
 # Vidyo:  https://v.mozilla.com/flex.html?roomdirect.html&key=UK1zyrd7Vhym
 # Phone:  +1 650 903 0800 or 650-215-1282 x92 Conf# 95312 (US/INTL)
 #         +1 800 707 2533 (pin 369) Conf# 95312 (US Toll Free)
 #         +1 416 848 3114 x92 Conf# 95312 (Canada)
 # IRC:    irc://irc.mozilla.org:6697/#qa

QA Org Items

  • New Hires & Interviews
    • item
  • Upcoming Events
    • item
  • PTOs

Discussion Items

  • Usefulness of qa-contribute mailing list -- too much spam recently; what is it used for again? (Aaron)
  • Test Day 1/20 Reflection (Jason)
  • (DavidB) Check out powerball and try a simple addon to earn points for bugs you file and comment on.
  • Follow up on team discussion that stephend started last evening
  • Virtual Machine image maintenance transitioning from Al to Geo. Want to try setting update schedule at every six months. Is this OK?
  • Has QMO handoff happened?

QA Team Updates

Desktop Firefox (Juan)

  • Fx.Next
    • Firefox 10 sign off meeting is today at 2:30 PM in WarpCore
      • If you know of any blocking issues please raise them to Juan, Marcia, or Anthony
  • Silent Update
    • Default Compatible Add-ons:
      • Huge thank you to every who helped with Default Compatible Add-ons testing; shipping for add-ons back to Firefox 4
    • Add-on Hotfix
      • Hotfix add-on is live on AMO for beta users, will enable us to simplify the chemspill process for *some* chemspills
  • Crash Investigation
  • Maintenance and Security (ashughes)
    • Firefox 3.6.26
      • Testing is underway, targeting push to Beta tomorrow, release on Tuesday

Browser Technologies (Tony)

  • Issues
    • Tracy taking Services QA Community stewardship.
    • Fennec Native is at risk for 1/31 beta date
  • Fennec (kbrosnan/aaronmt)
  • Sync Client (tracy)
    • Android Native Sync code drop .3 is nearly ready. It should go to m-c today or tomorrow.
    • Add-on sync is bubbling up the channels (Fx 11) gps has added a series of test to TPS for this feature.
    • Switching from Mozmill to Sikuli for UI automation of sync
  • Test pilot (tracy)
    • nothing to report
  • Sync Server (jbonacci, jrgm)
  • BrowserID (jbonacci, jrgm)
    • Trains continue to roll through
      • Train 17: Bug 713961 - QA and deploy BrowserID train-2011.12.28 to production
      • Train 18: Bug 715728 - QA and deploy BrowserID train-2012.01.05 to production
      • Train 19: Bug 719243 - QA and deploy BrowserID train-2012.01.18 to production
    • Big additions in the current and next train:
      • Support for primaries
      • Support for localization
      • UI changes
    • BrowserID team is actively pursuing leads to act as primaries (email domains, etc).
    • QA work continues towards client-side automation
    • QA is also working with Dev on front-end and back-end unit tests
    • Other upcoming activities include build-out and deployment of PHX colo
  • Jetpack(?)
  • Pancake (nhirata)
    • Infrastructure changes, UX changes still occurring with nothing to test on the front end.
    • Dria (Deb Richardson) is now officially PM
    • Talked to Browser ID/WebQA team to figure out how to best tackle the issue of testing Browser ID and having infrastructure to have selenium tests for Pancake to run daily.
  • webapis (jhammink)
    • presently main focus with these is now with marionette on B2G (P2), mainly
      • SMS
      • Telephony
    • Insofar as they tie into work with B2G
  • b2G (jhammink)
    • now building, test driving marionette against b2g on both devices and emulator; have gaia (frontend) working with marionette, python, and JS
        • coming soon - blog post coming for all of the above
    • Currently manually testing Telephony, SMS, GAIA UI, very hard!
    • Test plan for MWC congress demo (last week Feb)
      • test plan will cover mainly functionality (use-cases), (but also reliability, performance, security tbd)
    • attending B2G workweek in feb
    • Two new interns to onboard
    • wanna use Vlad Maniak as a resource - and he has expressed interest!

WebQA (Stephen)

Community (Marcia)

'

QA Automation Service (Henrik, David)

  • General
    • Selenium 2.17 has been released
  • Mozmill
    • Mozmill 1.5.8 release has been targeted for today (including support for running tests in parallel)

Project updates

CI system for on-demand Firefox builds (Geo)
  • Window systems turned out to be challenging re: setup
  • Currently finalizing job setup to get prototype fully rolled
CI system for daily and l10n Firefox Builds (Henrik)
  • System runs stable within the last week - no fall-outs from Pulse
  • Due to lack of documentation for Pulse some issues with queues had to be fixed
Automated testing for add-ons default to compatible
  • Currently a WIP patch is available for testing
  • Waiting for the final list of add-ons to test including platforms and the MPT machines
MemChaser extension
  • Need to finish up the logging feature before we can go public on AMO
  • Working with the JS team on getting an API for garbage collection implemented
  • Collecting ideas for a zombie compartment detector
  • Feedback from various groups is absolutely positive
Case Conductor
  • Doing preliminary testing on 0.8 (pure-Django) drop
  • Feature list for 0.8 up-to-date in Tracker
    • Everything above the ".8 Feature Complete" bar is expected to be in 0.8 release
  • On-track for 0.8 complete on 2/3
  • Q1/2012 Goals created.

Apps (David)

  • Test Day on 1/20 Results (All)
    • We built a well documented software project, with github pages, pydocs ..etc.
    • ~7 contributors, ran into issues getting code up and running
    • Gained insight on future direction of apps extension automation framework
    • Began shift to java + selenium + sikuli + jbehave.
  • Mochi Tests
    • Met with Clint, got details on mochi tests, how to write / where to check them in ..etc.
    • Began experimentation process
  • Mobile (Aaron)
    • Reviewing sites on native fennec, for possible Android soup from Webkit -> gecko move.
  • Goals
    • Seed new test framework
    • Prepare for MWC.

Notes, Takeaways and Action Items

Today

  • Add notes here

Ideas around structure for QA - no names attach - general buckets - general work needing to be done - not cast in stone - subject for talks in/out of - quickly go through it


high points:

Desktop - Jan 21st. - 2:30 warpcore: sign off on 10 - chase.com not working ex. try to catch it all - general web usage testing - if there is any bugs, is it blocking? - plans have been discussed - mitigating plans - so far, no wiki for it - Anthony on 3.6 - in the wiki

Mobile - risk for 1/31 - maybe adjusting the release date for Beta - Moving parts - reiterate try to use mobile and get performance in it, responsiveness, pulling up tabs, etc. - Services side - new work for back end - browserid: next 2 trains are big - b2g - goal for world congress - ux demo - how to flash gecko/gia on a device

WebQA - Affliates - new contributer - rajiv is doing onboarding - Mozillians - contributer Austin

Community - planning community goals - Matt approached bmoss for a headset for contributer - if you want to send a thank you/or tool for a contributer, we will find a way to help - community giving program - funding; something small; non specific, just for contributing/in the community; if we get into specifics then it lends to mercenaries. - studies on compensation. We don't want mercenaries we want community

Automation services - speed improvements due to support for running in parallel - memchaser

Case Conductor - 0.8 release - 1.0 release for the quarter - ~ 3/1 0.9 - middle of march stabilization - end of march 1.0 - small projects into case conductor

Note: - rules of snakes - don't send Bob in a knife to a gunfight - when you mean a ton of time... - how much are we going to save ahead of time - give data for the time, for the bug - if it doesnt make sense to them, you're not going to persuade them - give the next level of information

Raymond - took over QMO

Org: META groups: - Community (Marcia) - triage (bugs) : - help people log bugs - pick up the phone; help understand issues to get a solid bug - QMO

- Automation - tools : some set that make the right set of tools happen; QA/community can help implement, but no necessarily set it up (permenant) - tests : need people to actually write the tests in automated tools (nonpermenant/pull to various groups/everyone eventually get there)

  • Web - Correctness

App Store (UI) M. O. Mozillians Socorro

  • Server - Performance testing/Load testing/Failover testing/Crypto/Data Store

Sync Browser ID -> Services App Store Socorro

  • Client (Native; Mac, Win, Android)

Mobile, Desktop Apps B2G (might be different)

-> we stand behind it because we said it... => shell -> of passing it to waverly. -> at some point we have to say no. what willing to stop?

--> What's the problem that we're trying to solve? --> Learn to say "No"

- Automation Team versus QA? Probably no difference.

[Clear understanding what the load is / What's going to break?]

[Amount of work is limited -> get the automation; maintenance work, = > do it incrementally; maintenance]

[Testing/Security testing is in all three buckets]

[Security Team] [Taros : Snappy, Telemetry] [try to have a standardization - one off ]

[strattling buckets? Community and Web/Server/Client? ]


[ Be able to cross train and do what needs to be done; specialty seems broken, expertise should be there though]

[ Accountability for the tools? - make sure that the group is happy; tool is useful? - check back with the customer - break down of communication/management]

[ Do it through influence/not authority ]

[ Skill sets are different for Web, Server, Client, ex selenium for web, but not client; server doesn't necessary have a ui, JSON thingy, ]


Next Step is to figure out details

Last Meeting

  • Action Items
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/StaffMeetings/QA-Staff-01-25-2012&oldid=394012"