SHIELD Studies Dashboard

Basic automated analysis of SHIELD studies as they appear in the telemetry data source.

Initially we will set up discovery of new SHIELD studies, and for each SHIELD study we will generate two useful analyses. The first will examine the prompt to join the study, while the second will examine the effect of being in a study on retention.

For each analysis we break down the results by release channel, survey name (surveyID), and then variation (treatment group).

Gregg Lind has written up a notebook for us to use as a prototype.

impact

SHIELD studies take up to a week after the study is finished to yield useful information because of the time it takes to analyze the results. They report via a common ping format, and the analysis is largely the same between studies.

We can save a bunch of analyst time and provide early preliminary results by automating these analyses for each SHIELD ping that comes in through unified telemetry.

partners

• User Advocacy (Matt Grimes)
• Gregg Lind created the prototype analysis

artifacts

• code on github
• application on heroku
• [| other documentation]