QA/Browser Technologies/2011-11-03

From MozillaWiki
Jump to: navigation, search

« previous mtg | Browser Technologies Home | next mtg »

Discussion Items

  • Mobile
    • Amazon App store investigation
    • Test Strategy for Fennec Native 1.0. Use https://etherpad.mozilla.org/fennec-native-ui-qa-bugs for notes.
      • My ideas: Functional Testplan, Older devices compatibility, Website compatibility, feature breakdown, Schedule?, new litmus test repo, testday plan (focused one for internal first, external later), Automation section on Marionette
  • Services
    • JRGM Starts next monday
    • ChenXia and jrgrlicky started working on Native Fennec Support. Need to incorporate mobile testing into the plan
    • BrowserID deployment support plan? Mid december will be final release to millions.
  • Community Growth
    • How are we tracking with our community team goals?

Project Status

Fennec

Execution (kevin, aaron)

  • Nightly (10.0a1) status
    • Likely a XUL fennec 'maintenance' release
    • Chance that we might not do a 10 release
  • Aurora (9.0a2) status (merge to beta Nov 8th)
  • Beta (8.0) RC status (release next week)
    • Waiting on releng on beta 6 build 2
    • Release on Friday
  • Native Java UI status

Automation (martijn, John)

  • Crowdbot extension
    • Browserchrome and mochitests (subset of them) are working locally, need to figure out how to package them in jar format and run them from there
    • Logging is working
    • Will post an updated version later today
  • Proof of concept for functional automation
    • Clint/Trevor have a version of robotium working well enough to launch fennec, hit address bar
    • have native battery app
    • should be possible now to launch fennec and point to webapi battery app/compare values

Specialized (naoki)

  • Crash Reporting
    • This Week:
    • Next Week:
      • Crash Reporting needs to be separated out for Native Fennec vs XUL fennec Nightlies
      • some skiplist tweaks are still being done.
    • Blocked:
      • None
  • Memshrink :
    • Past Weeks: None
    • Next Weeks: still waiting for valgrind for android.
    • Blocked: Mike Hommey is working on that.
  • Newsletter
    • Past Weeks : Newsletter released on Tuesday
    • Next Weeks : Newsletter to be worked on
    • Blocked : None
  • Exploratory Testing:
    • Past Weeks: try IME build tested for alexp;
    • Next Weeks: try build for panning and zooming; nightlies testing
    • blocked : none

Sync

Client (Tracy)

  • weekly trains have been rolling but often delayed by issues found in testing.
  • train delays have kept devs from working on other projects/features. Devs have complained the code is comparable to a sieve. future rewrite?
  • respect back-off fixes have landed across the board (channels) to assist OPs if servers fall over again.
  • Need to add regular back-off testing into test plan.
    • weekly smoketest
    • a full functional pass 2 weeks prior to channel merges

Server (James)

  • Completed Trains
    • Bug 695564 - Deploy account-portal rpm-2.9-3 (along with server-core rpm-2.7-2)
  • Current Trains
    • Bug 694222 - Test and deploy MySQL option innodb_locks_unsafe_for_binlog + READ COMMITTED
    • Bug 693385 - deploy updated server-core (2.6.x) to syncstorage
  • Continued forward progress on Sync6 setup for general QA test, tools, development usage

Test pilot (tracy)

  • only thing slated is a new study on broken add-ons. jono to provide details when it is ready.

BrowserID (james)

  • Completed Train
    • Train 10: Bug 696415 - QA and deploy BrowserID train-2011.10.20 to production
  • Current Train
    • Train 11: Bug 697785 - QA and deploy BrowserID train-2011.10.27 to production
    • NOTE: This train is on hold pending debug/triage of a couple open issues
  • Continued forward progress on ID2 setup for general QA test, tools, development usage

Pancake (Naoki)

  • Current Pancake
    • Current
      • Documentation work and Process stuff being handled
      • iOS .17 version finished
      • old fxhome has new email - tested a little to see how old version of app handled email
      • recent upgrade to Python 2.7;
    • Next Week
      • Addon will be completed; need to test
    • Blocked by
      • Waiting for Addon to test

WebAPI (John)

  • Currently working APIs
    • Camera API
      • Testpage available
      • Mac, Linux changes landed; testing needed
    • Battery API
      • Testpage available
      • Native android app available (for automating comparison)
      • need to combine app + testpage, e.g. using robotium for one automation piece
    • IndexedDB API
      • Testpage Started
      • Possibly need more mochitest coverage; needs investigation
    • SMS API
      • Not landed yet
    • Vibrator API
      • Not landed yet
    • Other APIs
      • TBD
  • Testplan
    • Not available yet
  • Community
    • Plan is to involve community/testdays to bang on test pages when they are ready. This will require some rework/documentation to the pages themselves.
    • An additional plan is to involve the community in building/revamping test apps around the api. These can be also the basis for WebAPI tutorials, etc when they are ready.

Round Table

Notes

Action Items

  • mobile team to meet at 1:00
  • b-tech team: please try out crowdbot for fennec xul; and give feedback
    • martijn : send out link to the crowdbot to the team
  • tchung : talk about impact of other communities in the QA community meeting
  • tchung : give bullets for browser tech on visibility for outreaching; show communities for each individual goals for people.
  • tchung and mevans to talk about crowdbot off line
  • jbonacci : dashboard would be good idea. Spreadsheet is fine as well; relevant bugs, status, checklist items.

Raw Notes

- capture the notes and send them out to waverly and sumo
3 major bullet points
tony : good to be back
wait til 2nd half of the meeting to talk about mobile
fennec native meeting at the end.  James and tracy get back to other stuff if they want

Services talk : 
- John is starting next week; senior server side engineer to help james and tracy for test infrastructure 
- runner between ops and us

- the sync team is being allocated for native ui fennec
- there are a couple of wikis that they are going to figure out how to get sync support
- how to incorporate mobile and sync testing
- traditionally it's been driven by mobile to do it.
- with native fennec, we should
- sync is part of the first release
- sync team is working on the backend for sync, but not the ui.
- sync qa team knows sync but not mobile, and mobile qa team knows mobile, and some sync.
- something for tracy and james to think about in terms of backend infrastructure
- test matrix to including mobile, there are the plans for that currently, but because of the java the sync has to be redone because of the storage system is different
- cross team expertise is needed
- up to fennec 10 XUL is also being supported
- what's the best way to approach this?

- sounds like it needs to be in fennec 10 UI test plan for sync; major feature of the client
- the back end is changing per se.
- plan to be major integration within fennec (from mike conner)
	- concern with too much power to google in terms of information giving that
	- leaning towards leaning unified accounts
	- what if we introduce sync 
	- do we need gmail account?
	- more design questions
- fennec feature with sync developers working it
	- someone needs to drive it around ui fennec side w/ support from sync team (collaboration)
		- understand dependencies and architecture
		- ux people work with fennec
		- shouldn't matter who the contact person is
		- just need to figure out the distribution of people
	- back end is still traditional
- related project is the open web app
	- native fennec is going to support html shim
- need to look at scope of native 1.0
	https://wiki.mozilla.org/Mobile/NativeFennec1.0
- proposal :
	- XUL up to 10 : QA mobile team supports sync; tracy does verification with sync on mobile and desktop
	- Native : tracy work with kevin w/ new features ; feature test plan : the spec was sent out today

- browser ID support plan : 
	- talk to dan mills
	- plans to release ; don't have good grasp on scaling plan, staging, automation, etc.
	- haven't looked at plumbing plan.  James and Ioana have been working on client side bugs
	- deployment plan : the other pieces like unit test framework, Jeff's team talked to ops
		- they are patterning after the sync server
		- dev, staging and production enviroment.  Very similar to sync
		- the amount of client activity is going to slow down.  Most of the work is done on the server side
		- client is dummy web for the most part
		- testing will be more server side; log checking.
		- less client /ui testing
		- haven't seen the certification handling.
		- could be split into multiple services
		- change in production?
		- should be able to similate everything (load etc)  on staging
		- load test needs to be more related to browser id
		- don't know if it's created yet; tickets open for those tools; might not be the same thing
		- need to run functional test while doing load testing
		- make sure on our side, we don't have those final issues like we did with sync
		- lloyd having to separate things to have separate RPMs (still working on details)
			- splitting putting in different directories and then repackaging showed some issues
		- working with pete and monitoring the issues
		- any site that uses it, can point to it.  what we need is more sites.
		- need more real world sites using browserid
		- before we sign off on this.
		- is it pointing to our beta.  do we test those sites while generate load
		- we need both.  more sites ; more testing that doesn't relying on party and do testing on the server side
		- everything else you type up lives on the server
		- API testing, we don't have that yet.
		- John needs to work on API testing
		- talked to lyod ... need more sites ; QA verying
		- would rather have real sites.
		- "can you point to stage?"
		- externally there are some websites that use it.  such as Open Photo
		- yesterday or day before developer engagement : need to find 600 users
		- went do dev identity.
		- developer presentation outreach
		- we can get them involved.
		- silly dumb sites still need to exist (myfavoritebeer.org)
		- sync.in has more details
		- wiki site that has stuff listed
		- "does the environment work?" will be the first initial testing; make sure that it's stable first.
	- overlay is open web apps.  need to make sure that the environment works.  Make sure the train is there. and stable.
	- need a highlevel checklist.
	- happy to make a wiki for that.
	- some enumerated set of headings with "this has to work" and status
	- ex. load testing ; subtest under that
action item:	- jbonacci : dashboard would be good idea.  Spreadsheet is fine as well; relevant bugs, status, checklist items.
		- QA side meeting for the details; taking offline for QA centric
		- ping st3fan/nhirata for the staging pancake
	- services : reminder next week.
	- anything blocking bring it up.  desktop/mobile : go/no go later today (2:00)
	
Check with community team goals.  yesterday's meeting: we could be doing a better job.
	- community is a tough thing.  time , resources, ... asking a lot
	- james brought up a good point.  We are at the max
	- even so, just ignoring it or letting it slide... can't do that
	- "what can we do?"
	- we are conducting test days?
	- is there a better communication that we can do?
	- newsletter?
	- a lot of the things coming up, how do we use our community for testing?
	- not just on test days.
	- if it's open?  QMO?  Browser ID ... we need your help!  Come check
	- how do we entice people to help?  jobless friends?
	- bbd?  If you help build this, it would look good on your resume
	- web api stuff: leave it with the open question with "how can you make it better?"
	- give them a challenge.  Give them a carrot... that's going to help a lot.
	- "What's in it for me?" for the community ...
	- articulate that.  Here's what it is... here's the value
	- take away : first off Q4 goal.
	- community goal.  Target for each individual.  can collaborate that and put it up
	- we can make this more meta.
	- these are some things that we are trying to do.
	- for example : Naoki's newsletter.
	- test days, outreach events, mozcamps : and see a physical page?
		=> good start.  community is a promenent slice to show what we are doing
	- need more visibility : seeing what people are doing
action item : tchung : give bullets for browser tech on visibility for outreaching; show communities for each individual goals for people.
	- newsletter: feedback.  good stuff.  nightly testers ; we got gaby2300 testing outside of test day and some other mozilla workers
		- how wide: QMO, planet, Naoki's blog, mobile, browser tech
		- action item to go to about:mozilla
		- how do you get involved with it?  etherpad and Naoki announces.
	- how do we get outside in educations and such
	- asking about tools, jobs, etc. market place for fennec users.  We can definitely expand
action item: tchung	: talk about impact of other communities in the QA community meeting
	- crowdbot : martijn : browser chrome working and mochitests as well.
		- haven't gotten them packaged in the extension
		- working on right now
		- post an extension later today with package mochitest and browser-chrome test
action item: b-tech team: please try out crowdbot for fennec xul; and give feedback 
		- need to get this out next week for crowd
		- should we be spening resources to XUL base?
		- this will not die.  this is an addon; this is already in mochitest, gecko test, etc.
		- browser-chrome will not work, mochitest should work for native ui
		- we're almost there, we should finish it up
action item : tchung and mevans to talk about crowdbot off line
		- dev are mostly working native
		- results might get ignored for crowdbot
		- martijn to post link when it's done
- stop here for project status
- read the wiki through for each project status
- any project status.
- JHammick has project status : jonas's team working on ; integrated with b2g
	- can run with fennec
	- automation side proof of concept : ctalbert's team is taking over w/ robotium to support nativeui
	- John will drive later
	- has it been finalized?  no : just running as a proof of concept for now.
	- launching fennec; battery api is working and comparing with native battery app
	- what's the state w/ automation?
	- as we build up the abilities of the web apis.  How do we check them independently?
	- dealing with issue with accuracy of the geo location APIs?  We didn't see it.
	- this is an example where it would fail; proof of concept.
	- take a look at geolocation to see if we can do something like this
	- didn't list all APIs in the APIs section
	- it doesn't seem like it's overlapping with anything
	- it offers good automations in semi automated, where it could be crowdsourced as well as a full automation that it does the full life cycle where the data gets pushed to a server
	- Robotium route : we are pulling data out of the native API.  
	- we might be able to take the class and stick it in the automation
- 11:50 ; Mobile strategy
	- hard stop at 12
	- free at 1:30?  release go: no go is at 2:00.
	- place the meeting at 1
action item : mobile team to meet at 1:00
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Browser_Technologies/2011-11-03&oldid=364274"