Identity/Persona-Analytics
From MozillaWiki
< Identity
Last updated: 2013/11/22
Contents
Internal (Private) Metrics
We have two dashboards that support the #signin team (LDAP required):
- KPI dashboard (Maintained by Identity team)
- Identity dashboard (Maintained by Metrics team)
Public Metrics
- Persona related metrics collected by other sites: Third party metrics
Stakeholders
- Project Management: Are we growing? Are we succeeding at our goals? Do we have RPs in the pipeline? Are we converting them?
- UX: Do users proceed successfully through the dialog? Does UX A vs UX B improve our bounce rate?
- Developers/QA: What's breaking? Are the dialog response times acceptable? How frequent is a particular bug/issue?
Note: "Is the service up and functioning properly?" is not a question answered by these dashboards -- that is handled by monitoring, which is a separate function.
Meetings and Status
- Weekly meeting notes
- https://id.etherpad.mozilla.org/metrics-2013-08-19 Meeting with Annie about BI/DW
- A/B testing
- metrics futures discussion 6/17/13
- old status updates
Projects
A list of all Persona Metrics projects we'd like to do someday:
| Task | Description | Crew | Target |
|---|---|---|---|
| Two Metrics That Matter - KPI Dashboard | Two metrics that matter: one for RP adoption and one for user adoption. These metrics should be accessible in a usable (intelligibly badass) dashboard. | John Gruen (UX and dashboard coding), Ryan Feeley (UX), Katie Parlante (dashboard coding, lead), Shane Tomlinson (browserid data source) | Postponed |
| RP A/B Testing | Co-ordinating A/B testing hosted by RPs. Tests placement of initial Persona buttons/links/introductory text. | Ryan Feeley, John Gruen, Dan Callahan, Katie Parlante | Postponed |
| Dialog A/B Testing | Test different options on the first screen, optimizing to minimize bounce rate. Create our own infrastructure to run tests, as we don't want to use 3rd party options in our dialog. | Shane Tomlinson (browserid infrastructure), Katie Parlante (results display) | Unscheduled |
| Move KPI infrastructure to Ops team | Host KPI servers in VPC, instead of on ephemeral instances maintained by dev. | Gene Wood, Katie Parlante | Postponed |
| Self service access to data | Use RabbitMQ or Heka as transport, route to an elastic search backend, use kibana (or similar) as frontend | Unscheduled -- prototype might be good freaky project | |
| System architecture cleanup | Replace https (transport) with Heka/RabbitMQ and couchdb (data store) with elasticsearch . | Unscheduled | |
| Dashboard Improvements | Performance visualizations, email verification stats, etc. | Unscheduled |
KPI Dashboard
- KPI Dashboard tasks tracked in github: issue list
- Browserid, all github issues flagged as KPI: https://github.com/mozilla/browserid/issues?labels=KPI&state=open
- identity-ops github issues:
- make RP data available: https://github.com/mozilla/identity-ops/issues/102
- move kpiggybanks to production vpc: https://github.com/mozilla/identity-ops/issues/92
- Privacy Reviews:
- review kickoff for RP/email verification features: https://bugzilla.mozilla.org/show_bug.cgi?id=909980
- UX Redesign (TMTM): https://github.com/johngruen/kpi_des
Code & Issue tracking
Documentation
- https://www.lucidchart.com/documents/view/42df-90fc-5240a53c-8ea1-5c2f0a0089aa (Diagram)
- https://wiki.mozilla.org/Identity/BrowserID/KPI_Dashboard (a bit out of date right now, but the overall picture is right)
Identity Dashboard
Open bugs:
