Mobile/Metrics
From MozillaWiki
< Mobile
This page explains the different tools available to analyze data about how people use Firefox for Android and iOS. Most of these tools currently only support Android, but eventually there should be more support for iOS.
Note: some metrics are only available to employees and may require sign-in.
Contents
Usage Metrics
Dashboards
- Adjust: Third-party SDK initially added to track install referrers, but currently our most trusted source of usage numbers. Also used to track retention.
- ADI dashboard: Gives us more granular daily use numbers per release version. Very old way of collecting usage numbers, based off blocklist pings.
- iTunesConnect: Apple's metrics tools for iOS apps. Provided detailed information for installs, daily users, retention, etc. Some measures are "Opt-In" only.
- Users & Engagement: Based off core ping.
Telemetry
Our telemetry systems are currently in flux, but we are working towards a unified telemetry data pipeline to measure sessions and events. We have two levels of telemetry, opt-out and opt-in, which are controlled by data sharing preferences in our settings UI. Opt-in telemetry is enabled by default up to beta, but is really opt-in on release.
- Telemetry docs: Some of this is used on both desktop and Android, but some of it is specific to desktop only. Use caution when reading these docs in a mobile context.
- UI telemetry: Mobile-specific UI telemetry implementation. You can query this data with re:dash, or look at the UI telemetry dashboard. Learn more about creating probes in this guide.
- Core ping: New opt-out telemetry ping that we're working on to replace FHR.
Pipeline
Getting data into the system requires multiple steps:
- Client submits pings.
- Pings are generated and an attempt to send is made.
- Due to network connectivity, server issues or any other real-world issue, the ping could fail to be sent.
- Failed pings are archived locally and resend attempts are made at a later time.
- Server receives pings and does a small amount of validation and routing.
- Pings are ready for streaming style analyses. (~1 minute after receiving)
- Pings are saved in durable storage. (~15 minutes after receiving)
- Ping data is available to Spark analyses. (~15 minutes after receiving)
- Ping data is run through ETL scripts and imported into Presto/Re:dash. Ready for snapshot style analyses. (~24 hours after receiving)
Exploring Telemetry Data
- Histogram dashboard: Histogram probes are used mostly for engineering metrics. They are not as useful for product decisions, and we may stop sending histogram telemetry data on mobile.
- Self-serve telemetry analysis: IPython-based notebooks, great for doing more hardcore, specific telemetry analysis. Also used to create ETL scripts which load data into other systems.
- UI telemetry dashboard: Simple pivot-table summary of UI Telemetry. Re:dash can replicate the table, so we'll probably be moving away from this dashboard.
- Re:dash via SQL: Handy tool for performing analysis against telemetry data. Right now this includes opt-in UI telemetry data and core ping data. Learn more about the data structure and how to query in this guide.
Deprecated
- Executive metrics: Mostly not useful since we removed FHR, but a re-booted version is planned.
Stability and Performance
Dashboards
- Crash stats: Where we monitor crashes.
- Perfherder: Charts to monitor memory use, build times, and APK size.
- Autophone: Startup performance tests.
