Event Telemetry
The Telemetry wiki page has more information about using Telemetry -- this page describes the Event Telemetry project.
Contents
Overview
There is a common need across teams (fx-team, mobile, test-pilot, heartbeat, …) to have a mechanism for recording, storing, sending & analysing application usage in an event-oriented format. The Data Platform team wants to support this with a common API and mechanisms for dealing with the collected data, without owning the individual measurements. The solution here is to provide common client code, a standard data format, so we can come up with common processes and tooling for data pipeline & analysis work. Historically we already send a form of UITelemetry data, but the current format is too complicated to work with and to maintain.
Goals for Event Telemetry
- Standardized event format for all pings collecting events (Main Ping, Test Pilot, Shield, Sync, etc.)
- Expand collection of UI Telemetry to support Product teams
- Enable top line metrics for 2017
- Standardized support for analysis of events
- API support in client for collecting event telemetry
Documentation
Analysis and Reporting
- Raw data using a spark cluster (ATMO): https://analysis.telemetry.mozilla.org/
- re:dash event data tables (STMO): https://sql.telemetry.mozilla.org/
Project
Deliverables
- 2016 Q3
- Project kickoff
- Common Event Format design
- Event registration mechanism design
- Dataset strategy (Event Telemetry Data sets discussion)
- 2016 Q4
- Telemetry events on main pings implemented in Firefox
- Activity Stream submitting events (using common format, in main ping)
- Event data accessible in Spark
- Generic Event data table available in re:dash
- Test Pilot submitting events (using common format)
- Custom sync ping is submitting events (using common format)
- 2017 Q1
- More custom pings submitting events (using common format)
- Non-specialized, analysis-oriented datasets available
- Mobile plan (Q1/TBD)
- Some project (TBD) is on the main ping
Client work
Client Testing
Communication
- Conversation about Event telemetry on fhr-dev: https://mail.mozilla.org/listinfo/fhr-dev
- IRC: #telemetry
- Slack: #fx-metrics
- Weekly Meeting notes
- EPM reports
People and Roles
- Georg Fritzsche (Data Platform, lead)
- Alessio Placitelli, :Dexter (Data Platform)
- Mark Reid (Data Platform)
- Roberto Vitillo (Data Platform)
- Sunah Suh (Data Platform)
- Rebecca Weiss (PM)
- Ilana Segall (Analysis)
- John Dorlus (Quality Engineering)
- Thomas Huelbert (project management)