Feedback to Bug

Summary

Sometimes users leave feedback on Input that describes a particular bug and is detailed enough to serve as the basis for a Bugzilla bug. We want Input analyzers to be able to click a button on the feedback response view that opens up Bugzilla with a bunch of fields pre-filled with data from the feedback response. We also want analyzers to be able to find feedback that is considered to "likely" be a bug easily.

Status

• Project owner: None
• Status: Planning

History

• 2011: Tell Us More project is created to supplement the sad feedback form with an option for users to file Bugzilla bugs directly.
• December, 2015: Osmose is briefed on the project at Mozlando.
• December 18th, 2015: Osmose writes this initial project plan.

Phase 1

Summary

This phase will include adding a very basic "File a Bug" button to feedback pages, as well as making available a listing of feedback considered to be possible good bugs.

Requirements

1. Add a "File a Bug" button to the feedback response view that:
   • Opens up Bugzilla in a new tab with the fields pre-filled out based on the feedback.
   • Determines the correct product/component based on the Product the feedback was left in.
2. Add a listing (or modify an existing listing) to show feedback considered to be good bugs.
   • Optionally add the "File a Bug" link directly to this listing.

Input to Bugzilla Mapping

"Good Bug" Filter

We have an algorithm that analyzes feedback text and produces a "helpfulness" score measuring the likelihood of the feedback being useful or just hate speech. The "Good Bug" filter should take this and a few other things into account:

• Does it have a high helpfulness score?
• Does it have an email address associated with the feedback?
• Did the user provide about:support information with the feedback?

Measuring success

If we use a custom event in GA for when users click the file a bug button, we should be able to compare button clicks to views of the feedback page to measure engagement.

Question: Is the button being used?

This is measured by:

   clicked on the file a bug button
   --------------------------------
   total views of the feedback page

Our expectation is that the button won't be used very often, so we're not entirely sure what a goal is here. However, Phase 1 is low enough effort that measuring isn't too critical.

Question: Is the "Good Bug" filter effective?

This is basically asking how often feedback that is marked as a "good bug" is successfully converted into a bug. If we wanted to be super-accurate, we'd test a few different algorithms and compare their performance, but that is probably overkill in this case.

We could measure this:

   clicked on the file a bug button
   ----------------------------------------
   feedback was seen in a "good bug" filter

Which would give us a rough estimate of how often feedback that has been filtered was actually converted into a bug.

Bugs

Main bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1233866

Future phases

Future phases will be figured out after the most recent phase is completed so we have time to act on what we've learned.

Future phases may include:

1. Adding a new listing of feedback that shows feedback that are considered likely good bugs.
2. Searching bugzilla for bugs related to the feedback response.
3. Automatically generate the bug instead of pre-filling out the form.