QA/Fennec/Support Heartbeat in Fennec (Content Notification sentiment)
From MozillaWiki
Overview
Purpose
The purpose of this wiki is to serve as a general test plan for verifying that Fennec provide a feedback dialog when users turn off Content Notifications
Ownership
- Developer contact: X
- QA: Sorina Florean [:sorina]
Overall Status
[NOT LANDED] Nightly 50: NA
[NOT LANDED] Aurora 50: NA
[NOT LANDED] Beta 50: NA
Testing Summary
Scope of testing
The testing will be focus on
- verifying that users can provide feedback why they've turned off content notifications
- verifying that once feedback is submitted there is a chance to still receive a tip or hint or further information on the topic so that users might be able to change their option
Environments
Testing will be performed on following devices:
- Phones
- 4.x
- 5.x
- 6.x
- Tablet
- 4.x
- 5.x
- 6.x
Specifications
- As a Mozilla, we want to provide the user with a feedback channel to let us know why they purposely decided to opt-out of a feature
- As a Mozilla, we want to ask specific questions prompting this opt-out that will tell us why the user turned it off and if there is anything we can do to make the feature better
- As Mozilla, we want to collect and analysis the feedback about the opting out to make further product decision
- As Mozilla, we want to store this sentiment in the users profile so we can correlate this to further potential actions the user is doing, i.e. is she/he coming back to the future and turning it back on?
Requirements
- Include UI to start to showing and tracking heartbeat data inside Fennec
- Send a heartbeat ping with that data once the user turns off in settings, prompt them why they want to turn it off (canned answers)
- Content notifications as a our first run at Heartbeat with canned responses
User stories
- As a user, I want to provide feedback why I've turned off content notifications
- As a user, once I submitted my feedback would like to get a chance to receive a tip or hint or further information on the topic so that I might be able to change my mind (e.g. provide SUMO article)
Acceptance Criteria
- Heartbeat prompt shows up once user turns of content notifications under settings
- Heartbeat canned question and response is being presented to the user in an accessible way
- Heartbeat data is properly collected and accessible in re:dash
Success Criteria/KPIs
- Engagement rate: How many people turned off notification and did (not) react/communicates via Heartbeat prompt
- Product can decide what things are missing in the feature, what people frustrates about it and make product improvements, we could then run another heartbeat prompt on those users to check in and verify that the product has now become more useful to them
- Report that outlines most common reason users suppressed notification is X, Y, Z, which implies we need to improve the product in direction A,B,C (see Google doc table)
Additional Info
- Google doc with canned answers
Testing details
Test Cases
Testing days
Date: 2016-07-21
- Created Test Plan
Bug Work
- Meta bug
- bug 1272717 - (heartbeat-content) [meta] Heartbeat for Content Notifications
ID | Priority | Summary | Status | Assigned to |
---|---|---|---|---|
1272719 | -- | Add feedback dialog for turning off Content Notifications | RESOLVED |
1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);
Still issues
[NEW] Bug 1272719 - Add feedback dialog for turning off Content Notifications
Bug fixed
NA
Signoff
Criteria
- All test cases should be executed
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed
Results
Nightly testing NA
Aurora testing NA
Beta testing NA