Firefox OS/Performance/Boot Timing Automation
As per Bug 1045938, the plan for implementing automated boot info capturing will be described below. The end goal is the eventual creation of a system to support the continued tracking of the status of boot performance. That is to say, one should be able to simply do something like
make test-perf boottest
and receive the appropriate data. Furthermore we would like to get this information into datazilla along with all the other performance numbers we track.
Plan Outline
This primarily relates to bug 1047118. The simplest approach we could take for accomplishing this is as follows:
The data emitted for each stage will be:
- Capture and store pre-gecko timing to disk (something like
"/data/local/.BOOTTIMES.json")- See bug 1058810 and bug 1058811 for implementation.
- Currently setting an android property in init, and accessing it from b2g.sh to write out to disk
- Also capturing the timing of when we launch b2g proper
- File in JSON format to be compatible with the other tools.
- settings.js/shell.js generate a time stamp (that is stored in some standard property of the window) near its entrypoint.
- See bug 1058817 and this changeset for proposed implementation.
- System app (in chrome) records timing at some decided location (as in the prior step):
- Do this right before iframe creation in shell.js ?
- See bug 1058817 and this changeset for proposed implementation.
- System App:
- Vertical homescreen
- Record time stamp at entry point
- See implementation here.
- moz-* are emitted
- There is no way of actually observing these events for homescreen currently implemented, and there needs to be work done to make use of these events being fired in this test framework.
make test-perf will consume the .BOOTTIMES.json file, and all the properties to give back meaningful data. See bug 1058843 and this changeset for the boot test's implementation.
In the pre-gecko timing one of the data points will be a pair of both "uptime" and "epoch time". Using this, we can calculate the time of boot in terms of epoch time, which is more compatible with times we have access to in the higher levels of the stack. If there were ever to be some WebAPI to get device uptime, this would be a far preferable means, and would very much minimize the complexity currently needed in this implementation. .
Another problematic portion of the implementation is that as a result of the system date taking time to correct itself upon boot (sometimes this is even after gecko has started), depending on when one samples the date, it is possible to get a very incorrect date. As a stop gap measure, I collect a time "early" in the boot sequence, and another "late" in the boot sequence, and for the calculations use which ever is more appropriate as the reference.
In the current design of make test-perf's startup_event_tests, the test designates code to be injected, and launches the app, thus having full control over the whole process. This is not quite a suitable approach for the system and verticalhome apps which are kind of special cases. Upon gecko finishing its initialization steps, shell.html is unconditionally loaded, which loads the system app directly (which then also goes and prepares the lockscreen and the homescreen). The approach of injecting certain handlers may be possible however, if one reloads b2g to give the code a chance to be injected into the relevant frames.
What the boot_test does is, instead of firing and catching events, it (in the future perhaps conditionally upon a setting) records time stamps in some predetermined object for the test harness to collect afterwards. This way there is minimal impact on the app's performance. The test looks for results in the system chrome, the system app, and the homescreen.
Data Visualization
Once the data itself is collected it could be presented in datazilla. What this would look like would be for there to be a Boot test, and for each stage in the boot process to have its own line tracked over time. The Y-axis would be the cumulative time from boot, of each stage. This allows one to easily see how the duration of each section changes over time. Some parts (like the kernel) would likely stay relatively static, while we may see more fluctuations in others (gaia or gecko for example).
