Criteria

Definitions

First Launch 

Milliseconds elapsed from the user-initiated launch of a not-already-in-memory app to the stable display of that app and its visible content. These timings are captured to the moz-app-visually-complete event documented on MDN.

First Launch Memory 

Maximum bytes of RAM used by an application after completing its First Launch.

Reference Workload 

A standardized collection of application data preloaded before the test is run.

Note: All measurements are the median results calculated from multiple iterations of relevant tests. For example, Datazilla's App Launch values are calculated from 30 such iterations.

Reference Devices

• Flame
• Sony Xperia Z3 Compact (Aries)
• Others phone

Application cold launch

FxOS 2.5

• Reference Device: Flame-KK 319MB and Aries-KK
• Reference Workload: Light

NOTE: These apps have been instrumented to emit events that support timing their launch states.

FxOS 2.0, 2.1, 2.2

• Reference Device: Flame 319MB
• Reference Workload: Light

• For Camera App, as discussed with the PM, the performance boundary will be based on the average of 3 performance runs done in Oct 2nd, 21st, and 30th. If the number regresses more than 100ms from the criteria, it will be considered as a regression.

FxOS 1.4

• Reference Device: Buri/Hamachi
• Reference Workload: Medium

FxOS 1.3T

• Reference Device: Tarako

Acceptance for 1.3T will be determined by reviewing final performance results which will include cold launch time.

Memory Usage

Power Consumption

MTBF (Mean time between failures)