Perf-O-Matic 2.0

• Perf-o-Matic staging

• hg.m.o/graphs

• Implement new graphserver interface - bug 601796

Problem?

File a bug in product Webtools component Graphserver version 2.0

/connect irc.mozilla.org

/join #perfomatic

Design

Designed by Chris Howse

Original UI Prototype

Missing features from current design

• dashboard ->
  • toolbar ->
    • product - not implemented
    • platforms - not implemented
    • top tests - not implemented
    • set pulldowns based on URL
  • data grid ->
    • row/column should not scroll
• custom chart ->
  • toolbar ->
    • running time - time delta/time delta % not implemented
    • show changesets - needs design work
      • may need hgpushlog work to link directly to changesets
    • export CSV - server broken bug 489775
    • link - needs url shortener
      • url/embed styling prototype changes
    • embed - not implemented
  • set selection range based on URL
  • add test data ->
    • put popular branches/tests/platforms on top
• plot ->
  • y-axis should always be 0-based
• need mapping between branches and hg repos

Performance optimizations

• split up JSON files as discussed in status meeting
  • look into constraining by date range
• mini-manifest, return only requested testid/branchid/platformid
  • e.g. http://graphs-stage2.mozilla.org/api/test?attribute=short&testid=1&branchid=1&platformid=1
• set Last-Modified headers on all JSON feeds
  • especially the manifest!
  • already setting 15m Expires/Cache-control max-age
• add caching proxy (varnish, etc)
• improve charting performance
  • forcing lineWidth to 1 seems to make a huge difference
    • started discussion on the flot list, fixed in flot and backported to graphs flot 0.6
    • however does not look as good, not pushing to stage until further investigation
  • "double-buffering" canvas for flot
    • stack overflow discussion
    • custom chart is much faster if each series is written to a hidden canvas then shown when they are all complete
  • use node.js to generate static images for dashboard?
    • flot group post
      • this actually works pretty well
      • spoke to author (John Schulz, jsno.de/) who linked the code behind http://jsno.de./flot/
      • requires a text plugin for flot, to get the ticks into the canvas instead of flot's default of appending styled divs
  • look into reducing number of points
    • flot plots way more than is actually useful when graphs are very dense, flot group discussion
  • look into flot alternatives
    • highcharts
    • RaphaelJS / gRaphael
    • Current graphserver library

Testing

• stand up vhost with old graphserver, pointing at nightly database, for QA comparison
• set up lint and tests on Hudson
• selenium tests (in progress)
• qUnit/env.js unit tests for JS code
• stand up auto-update for code (5 minutes)
• stand up auto-update for DB (nightly)
• (QA) compare actual submitted data to JSON, and confirm in interface
• make sure proper changeset range is shown for checkins

Bugs

• custom chart ->
  • bug 643892 "show changesets" with multiple data series loads multiple hg.m.o tabs
  • embed ->
    • need loading indicator for individual series
    • tooltips can be hidden by top of screen
  • series controls change plot daterange
    • str: add multiple data series and change daterange pulldown before all are loaded
  • selection range set when "sel=none"
• charts (general)
  • bug 643897 all axes should be labeled, units should be specified
  • bug 651690 y-axis is relative to entire dataset, not just current view
  • selection range ("sel" in URL) is very confusing if it is not in current view, or older than 365 days
    • could check this in JS, error if it's too old, and select 365-day view if it's not?
• "add more test data" dialog
  • bug 641414 branch, test, platforms should be sorted

Discussion

• reddit /r/programming thread
  • label axes (see Bugs section)
  • areweslimyet.com launched by jrmuizel ;)
• Hacker News thread
  • missing data from other browsers
    • how can we instrument Chrome, Safari, IE, Opera, etc.?
      • can we publish this data?
• old-style URLs are not compatible, primarily because the new graphserver automatically shows the average of all machines in a platform rather than a separate line for each, and the old-style URLs refer to individual machines. Problem?