Auto-tools/Projects/B2G Test And Performance

From MozillaWiki
Jump to: navigation, search

B2G Test and Performance Automation

These are our current projects to support B2G and other mobile efforts going forward.

Marionette

Marionette is a low-level test automation integration into the Gecko Platform. It will communicate via a JSON API that is captured by the JavaScript remote debugging socket. And it will interpret those JSON commands into actions via the accessibility (a11y) interfaces already present in the platform.

Eideticker

Eideticker is a video capture and analysis tool for performance analysis. It can also be used to do performance testing for fennec and desktop browsers as well.

Benchmarks

We'd like to address the wishlist of benchmarks summarized below (originally from this thread, and emailed suggestions, aggregated here):

  • Some brand name companies may ask for the performance of file reading/writing on different file sizes and file numbers.
  • [sicking] App startup time. we should start with the most critical built-in apps (phone, sms, browser), we should also do 3rd party apps, both packaged and hosted ones
  • [suggested by ieee engineer, off-thread] Boot time as a metric. This is specially interesting for the pandaboard user group, aiming to use B2G in OEM applications.
  • [Michael Treese, gal] Battery estimates. Example profiling:
    • Standby time (important), then videos and apps
    • Active radio usage (calls, etc) is for OEM to optimize
    • Test Battery life with other applications - Browser, FM Radio, FB, etc.
    • how many amps an applications consumes during normal usage
  • [overholt] Can we use powerTOP for power consumption usage?
  • [akeybl]:
    • startup/shutdown time, cold app launch time, warm app launch time (?), WebGL tech demo framerates, JS perf tests, dropped frames in video playback, memory usage all the while, and app eviction frequency (?).
  • [khu] CPU usage and memory usage:
  • [jlebar] Perhaps we can do a straightforward measurement similar to AWSY (except with apps instead of webpages), if we're running on an emulator with unlimited memory. If OTOH we're emulating a device with a fixed amount of memory, we'll have to be more clever in how we measure, since B2G will happily kill background apps when it's running out of memory. Maybe you can come to a MemShrink meeting and we can talk over how we might want to do memory benchmarks on the device. Don't be a stranger.

Meetings

Join us, we'd love the help! You can either drop by in #ateam (irc.mozilla.org) or drop into one of our meetings.

When: Every Thursday 10AM PDT

Where: Phone: 1-650-903-0800 x92, Conf #95589 OR 1-800-707-2533, PIN: 369, 95589

Etherpad: Jot down meeting notes here

Meeting Template

Create a new weekly agenda from the active Meeting Template:

<createbox> align=left type=create preload=Auto-tools/Projects/B2G_Test_And_Performance/Meeting_Template default=2024-12-01 prefix=Auto-tools/Projects/B2G_Test_And_Performance/Meetings/ </createbox>

Retrieved from "https://wiki.mozilla.org/index.php?title=Auto-tools/Projects/B2G_Test_And_Performance&oldid=487923"