Unified Telemetry/Status reports/Android/March 25 2016
From MozillaWiki
Contents
UT for Android status report March 16, 2016, 2015
Overall Project Health
Last week: Green
This week: Yellow
Project summary
- Unified Telemetry now in production - validation efforts show a large difference between actual and expected data. This is our current focus
Exec Summary
- Q1 deliverable is unified telemetry in android
- Ratio (MAU/DAU) is currently planned on being calculated by use of Adjust in Q2 (see KPI report for more info)
Accomplished for Last Period
- Release issues:
- Low number of clients, bug 1257589
- profileDate missing for some clients, bug 1257595
- Shortcomings from original implementation: https://bugzilla.mozilla.org/show_bug.cgi?id=1246816
- searchDefault is in review, bug 1249288
- Integration tests in review, https://bugzilla.mozilla.org/show_bug.cgi?id=1249491
- Fennec-dashboard data export in PR
Planned for Next Period
Validation:
- Investigate delta between expected and actual data
Risks and items to retire
Dashboards and consumers:
- Dashboard form core pings (need min requirements)
- PM input
- Significance of abnormal session and search counts for first rev.
Validation:
- automated tests
- manual QA
- mobile team has in-house and SoftVision QA
- scope of v0 ping version is not be appropriate for heavy test effort
- manual QA will happen from v3 on
- Android Nightly has small population, so:
- Sanity-check on Nightly
- use Beta for validation
- update latency to beta ~2 weeks to 90% of population
- server-side analysis
- quality checks for ordering of pings, etc. and verifying the data looks sane
- sanity checks at volume (look for large gaps etc)
- validate against adjust data?
- we should get to an ongoing "health of data" report from there
- QA & analysis work needs to be defined and scoped/packed
- schema validation
- pipeline-side schema validation support is on the way
- pipeline team may be interested to start this with the "core" ping
