Test Pilot/2017Q3
From MozillaWiki
Test Pilot 2017 Q3 Objectives and Key Results
| Final Status (Sep 30) | Comments | ||
|---|---|---|---|
| Objective 1 | Improve storytelling so that we support Firefox and grow community | .48 | |
| KR1 | Bi-weekly updates for each new experiment published on the blog or experiment pages. | .66 | We didn't publish updates at this cadence for every experiment, but we did publish a lot of things, including specific posts on the experiments. We decided this was about 2/3rds the spirit of this KR. |
| KR2 | Get 1000 combined reads on all blog posts | 1 | So many reads. Over 6000 near the end of September. |
| KR3 | Have 15 people participate in user research on design prototypes | .27 | Only 4 participants in the Notes optimization study |
| KR4 | 30 attendees at a Brown Bag about storyboarding | 0 | This was planned for Taipei but that trip was cancelled. We've made a new plan to host a meeting in Austin with the ET team. |
| Objective 2 | Grow Screenshots usage and expand offered features so that people find Screenshots to be a useful tool. | .73 | |
| KR1 | Ship screenshots to 100% of release users | 1 | 🎉 Shipping by default in 56 |
| KR2 | 10% of Screenshots users sign in to FxA | 0 | A patch to integrate FxA exists, but we didn't ship it to production. |
| KR3 | .5% of non-owner, non-Firefox visitors click a branded CTA | 1 | (total click events)/((unique non-ff visits to the home page)+(unique non-ff shot page views)) = 1.04% |
| KR4 | All pages and API endpoints load in under 2s (minus network time) | .9 | All the endpoints are well under 2s now, except we found the "delete all data" endpoint near the end and it's still pretty slow. |
| Objective 3 | Improve Test Pilot so that the Test Pilot program is a pleasure to use, conforms to the Firefox 57 plan, and improves engagement with experiments. | 1 | |
| KR1 | Ship SDK-free Test Pilot to 100% of users | 1 | |
| KR2 | Use storybook for all newly implemented feature cards | 1 | |
| KR3 | Implement 4 Test Pilot feature cards | 1 | 2 shipped, 2 waiting on L10n freeze and will ship next sprint |
| KR4 | Decrease home page load time by 25% | 1 | Beat this by a decent amount, but need to get the actual numbers from Donovan. |
| KR5 | Send a notification through the Test Pilot add-on using a WebExtension experiment API | 1 | This code exists and works, but didn't ship to production yet. |
| Objective 4 | Find, guide, and graduate ideas and experiments into features so that Test Pilot is a proven valuable input to the future direction of Firefox | .17 | |
| KR1 | Publish conclusions from three A/B tests for Firefox Send | .33 | We shipped one A/B test and published the results. |
| KR2 | Create a document with a plan to ship a partner-repack experiment | 0 |
