Mobile/Testing/03 20 13

From MozillaWiki
Jump to: navigation, search

Previous Action Items

  • [gbrown & jmaher & ted] investigate replacing libc and libdvm with custom compiled versions we have symbols for
    • ted is having trouble sync'ing the repos for the revision we run
  • [wlach & clint] determine priority for comparative tests
    • DONE. Conclusions:
      • (1) it's hard to come up with truly objective comparative tests. we have to be careful about managing expectations. [wlach] drafted a blog post on this, deferred posting because it links to the eideticker dashboard, which is currently down... but hopefully that will be a good starting point to future discussion.
      • (2) it should be possible (and useful) to do a comparative page load test between chrome/stock/fennec on the galaxy nexus, so we'll make that a priority. see bug 852744

Status reports

Dev team

  • Progress on Robocop intermittent failures
    • bug 797615 Intermittent testHistoryTab... | PaintExpecter - blockUtilClear timeout (x3) -- :kats determined cause
    • bug 851861 Intermittent testOverscroll... | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)}} -- :cwiiis is working on it
    • bug 814282 Intermittent testVkbOverlap | Found 0 green pixels after tapping; expected 2428 -- :gbrown is making progress

Rel Eng

  • (Callek - progressing on debug/xpcshell tests, debug looking likely to be ready first
  • (kmoir) - heads down on converting panda tests to mozharness so we can use mozpool

IT

  • Still working on a higher density chassis.

A Team

  • tegra failure rate [8.7%]
    • m1, m2
  • panda failure rate [12.3%]
    • rc1, rc2, m8
  • 3 softvision robocop test patches landed
  • still investigating pandas rebooting in the middle of the tests, will be working closely with dividehex on Friday

x86 automation

  • ctalbert is looking into it this week.

Autophone

We've temporarily suspended the production reporting of Autophone tests while we test the effects of the following:

  • bug 850799 - Bug 850799 - [s1s2] Run tests with and without cache enabled
  • bug 851288 - Bug 851288 - Investigate using quitter.xpi for terminating fennec in autophone s1s2 tests
  • bug 849251 - Bug 849251 - Further decouple the build cache
  • bug 852842 - Bug 852842 - Disable crash reporter ui in Autophone

Test results are being reported to http://mrcote.info/phonedash_bc/.

We are investigating the differences between test results and the production results on http://mrcote.info/phonedash/:

  • No January 16 regression
  • Not as dramatic an improvement of cached values over uncached and in some cases worse cached performance.

Questions:

  • Is cached vs. uncached performance expected to be significant? Do slow sdcards impede the efficacy of the cache?
  • Previously we terminated fennec by killing it. Now we use an extension that calls quit from within the browser. Is it expected that killing fennec in an unclean fashion caused the previous behavior?

We expect to have the production system back up and current within a few days.

Eideticker

  • Fixed various issues with phones/dashboard which were preventing regular updates, which made things run smoothly. Except...
  • Dashboard currently down, AWS instance backing it accidentally blown away yesterday. Whoops. In process of recreating it, should be up later today / early tomorrow.
  • As mentioned above, planning to create a cross-browser page loading test. See bug 852744

Round Table

Action Items

Retrieved from "https://wiki.mozilla.org/index.php?title=Mobile/Testing/03_20_13&oldid=639479"