Firefox/Activity Stream/OKRs/Desktop OKRs/2017Q2 Archive
From MozillaWiki
< Firefox | Activity Stream | OKRs/Desktop OKRs
Activity Stream - Firefox Desktop - OKR Progress Report for Q2 2017
Updated June 30, 2017
Latest Activity Stream Program Status Report
OKR Progress
OKR DOCUMENT HERE
On track, issues well managed - Confidence level 70% or higher
At risk, unknowns without clear mitigations - Confidence level 40-69%
Blocked on deliverables, needs decision, escalation required - Confidence level below 40%
| Objective | Key Result | Confidence | Notes | |||
| May 2 | May 16 | June 2 | Score | |||
| 1. ACTIVITY STREAM ENGAGEMENT - Activity Stream newtab presents users with a more engaging experience | 1.1 Activity Stream has feature parity with existing about:newtab and about:home | 75% | 75% | 40% | Final Score 0.44 | Currently at 14 Open and 11 Closed Parity Bugs Here. Did about:newtab but not about:home. Rating as attempted. 0.44 score. |
| 1.2 Activity Stream supports recommendations through Pocket API that have acceptable engagement measures | Final Score 1.0 | Currently at 14 Open and 11 Closed Parity Bugs Here. Remaining Open are part of MVP Scope and still considered on track for FF57. There is work that doesn't make sense to do on the Test Pilot side, and should only be implemented in the system add on, and all of this likely won't be completed in the system add-on by the end of June. | ||||
| 2. ACTIVITY STREAM PERFORMANCE - AS will not negatively impact Firefox desktop performance. AS will comply with all Quantum perf targets wrt Smoothness, Responsiveness and Page Load. | 2.1 Activity Stream is continuously profiled and optimized on a shitty Windows laptop | 60% | 60% | 60% | Final Score 0.7 | A certain amount of profiling is now happening, as the team does have access to 'shitty windows laptops' now, but can't quite be classed as 'continuous'. Rating as Satisfactory. Score 0.7. |
| 2.2 The Activity Stream system add-on collects and reports key (well defined) performance metrics through telemetry | 75% | 75% | 85% | Final Score 1.0 | Key telemetry infrastructure has landed and investigation into key metrics has been completed. We are currently collecting/reporting one key metric, so the hard work is done. Achieved what we needed to for this quarter. 1.0 | |
| 2.3 The Activity Stream system add-on does not regress Page Load, Smoothness or Responsiveness in Firefox relative to Tiles | 75% | 75% | 75% | Final Score 1.0 | Currently Activity Stream in Nightly meets and/or exceeds all performance expectations. Complete 1.0 | |
| 2.4 The Activity Stream system add-on does not cause appreciable main thread hangs | 75% | 75% | 78% | Final Score 1.0 | Activity Stream has been instrumented with a BackgroundHangReporter such that we will receive reports if any notable hangs occur, and thus can be investigated and resolved as necessary. Considering this as an 'evergreen' KR as this will be monitored on an ongoing basis. Complete. 1.0 | |
| 3. GRADUATION - Activity Stream experience is available to a meaningful percentage of the general Firefox Nightly population | 3.1 Activity Stream has landed in m-c and is pref'd on for at least 10% of Nightly users | 75% | 75% | 75% | Final Score 0.7 | Requirements for this KR were not all met and we are not yet pref'd on in nightly. Scope turned out to be larger than expected, and is still being worked through. Expecting to pref on in early July but not meeting the goals of Q2. Satisfactory. 0.7 |
| 3.2 Activity Stream has completed all privacy and security reviews/audits | 75% | 75% | 75% | Final Score 0.75 | Making progress and on track. Both groups have been engaged. Fairly confident that we are not doing anything (at least obviously) that would cause us to fail either review, but the review process is still ongoing. Satisfactory 0.75 | |
| 3.3 Activity Stream has passed all Accessibility reviews | 60% | 60% | 70% | Final Score 0.0 | Decided to do this once pref'd on in nightly, which has not yet happened. Deprioritized 0.0 | |
| 3.4 Ability to store and query Activity Stream specific metadata in PlacesDB module | 75% | 75% | 80% | Final Score 1.0 | Landed. Further details can be found in the bug HERE. Complete 1.0 | |
| 3.5 The Activity Stream system add-on sends all event telemetry to ping-centre | 75% | 75% | 90% | Final Score 1.0 | Complete. 1.0 | |
| 3.6 Sentry | 75% | 75% | 75% | Final Score 0.0 | There is a patch for this already. Approval to land this in Nightly has been given. Deprioritized. 0.0 | |
| 4. EXPERIMENTATION & ANALYSIS - Our capability to rapidly and accurately perform data based product experimentation is standardized and automated. | 4.1 Launched at least one experiment using production Experimenter deployment | 75% | 75% | 85% | Final Score 0.7 | Old experiment data has made it in there. The next new experiment will be launched via Experimenter, but did not make the end of the quarter. Satisfactory 0.7 |
| 4.2 Run Shield Study to prove AS is as engaging as existing about:newtab | 75% | 75% | 80% | Final Score 1.0 | Study is currently running. Just waiting for the data to roll in. Complete 1.0 | |
| 4.3 Build out experimentation infrastructure in Nightly and run at least one experiment | 60% | 60% | 60% | Final Score 0.0 | The requirements here have not been clear and thus nobody has been driving it forward. Dependent on being pref'd on for some percentage of users first so we have data. Not considered a priority by the team for this quarter. Deprioritized 0.0 | |
| 4.4 Deploy automated dashboard generation | 75% | 75% | 85% | Final Score 1.0 | 100% Test coverage. Complete. 1.0 |
