Firefox/Activity Stream/OKRs/Desktop OKRs/2017Q2 Archive

From MozillaWiki
Jump to: navigation, search

Activity Stream - Firefox Desktop - OKR Progress Report for Q2 2017

Updated June 30, 2017

Latest Activity Stream Program Status Report

OKR Progress

Green-sm.jpg On track, issues well managed - Confidence level 70% or higher
Yellow-sm.jpg At risk, unknowns without clear mitigations - Confidence level 40-69%
Red-sm.jpg Blocked on deliverables, needs decision, escalation required - Confidence level below 40%

Objective Key Result Confidence Notes
May 2 May 16 June 2 Score
1. ACTIVITY STREAM ENGAGEMENT - Activity Stream newtab presents users with a more engaging experience 1.1 Activity Stream has feature parity with existing about:newtab and about:home 75% 75% 40% Final Score 0.44 Currently at 14 Open and 11 Closed Parity Bugs Here. Did about:newtab but not about:home. Rating as attempted. 0.44 score.
1.2 Activity Stream supports recommendations through Pocket API that have acceptable engagement measures Final Score 1.0 Currently at 14 Open and 11 Closed Parity Bugs Here. Remaining Open are part of MVP Scope and still considered on track for FF57. There is work that doesn't make sense to do on the Test Pilot side, and should only be implemented in the system add on, and all of this likely won't be completed in the system add-on by the end of June.
2. ACTIVITY STREAM PERFORMANCE - AS will not negatively impact Firefox desktop performance. AS will comply with all Quantum perf targets wrt Smoothness, Responsiveness and Page Load. 2.1 Activity Stream is continuously profiled and optimized on a shitty Windows laptop 60% 60% 60% Final Score 0.7 A certain amount of profiling is now happening, as the team does have access to 'shitty windows laptops' now, but can't quite be classed as 'continuous'. Rating as Satisfactory. Score 0.7.
2.2 The Activity Stream system add-on collects and reports key (well defined) performance metrics through telemetry 75% 75% 85% Final Score 1.0 Key telemetry infrastructure has landed and investigation into key metrics has been completed. We are currently collecting/reporting one key metric, so the hard work is done. Achieved what we needed to for this quarter. 1.0
2.3 The Activity Stream system add-on does not regress Page Load, Smoothness or Responsiveness in Firefox relative to Tiles 75% 75% 75% Final Score 1.0 Currently Activity Stream in Nightly meets and/or exceeds all performance expectations. Complete 1.0
2.4 The Activity Stream system add-on does not cause appreciable main thread hangs 75% 75% 78% Final Score 1.0 Activity Stream has been instrumented with a BackgroundHangReporter such that we will receive reports if any notable hangs occur, and thus can be investigated and resolved as necessary. Considering this as an 'evergreen' KR as this will be monitored on an ongoing basis. Complete. 1.0
3. GRADUATION - Activity Stream experience is available to a meaningful percentage of the general Firefox Nightly population 3.1 Activity Stream has landed in m-c and is pref'd on for at least 10% of Nightly users 75% 75% 75% Final Score 0.7 Requirements for this KR were not all met and we are not yet pref'd on in nightly. Scope turned out to be larger than expected, and is still being worked through. Expecting to pref on in early July but not meeting the goals of Q2. Satisfactory. 0.7
3.2 Activity Stream has completed all privacy and security reviews/audits 75% 75% 75% Final Score 0.75 Making progress and on track. Both groups have been engaged. Fairly confident that we are not doing anything (at least obviously) that would cause us to fail either review, but the review process is still ongoing. Satisfactory 0.75
3.3 Activity Stream has passed all Accessibility reviews 60% 60% 70% Final Score 0.0 Decided to do this once pref'd on in nightly, which has not yet happened. Deprioritized 0.0
3.4 Ability to store and query Activity Stream specific metadata in PlacesDB module 75% 75% 80% Final Score 1.0 Landed. Further details can be found in the bug HERE. Complete 1.0
3.5 The Activity Stream system add-on sends all event telemetry to ping-centre 75% 75% 90% Final Score 1.0 Complete. 1.0
3.6 Sentry 75% 75% 75% Final Score 0.0 There is a patch for this already. Approval to land this in Nightly has been given. Deprioritized. 0.0
4. EXPERIMENTATION & ANALYSIS - Our capability to rapidly and accurately perform data based product experimentation is standardized and automated. 4.1 Launched at least one experiment using production Experimenter deployment 75% 75% 85% Final Score 0.7 Old experiment data has made it in there. The next new experiment will be launched via Experimenter, but did not make the end of the quarter. Satisfactory 0.7
4.2 Run Shield Study to prove AS is as engaging as existing about:newtab 75% 75% 80% Final Score 1.0 Study is currently running. Just waiting for the data to roll in. Complete 1.0
4.3 Build out experimentation infrastructure in Nightly and run at least one experiment 60% 60% 60% Final Score 0.0 The requirements here have not been clear and thus nobody has been driving it forward. Dependent on being pref'd on for some percentage of users first so we have data. Not considered a priority by the team for this quarter. Deprioritized 0.0
4.4 Deploy automated dashboard generation 75% 75% 85% Final Score 1.0 100% Test coverage. Complete. 1.0