Labs/Site 2.0/Existing site replacement/Site content/Test Pilot

From MozillaWiki
Jump to: navigation, search

Project planet feeds

None yet.

Download link

Link for downloading most up-to-date version of the software (if any)

  • N/A (will be released soon)

Discussions feed

Feed URL for latest discussions (new topics + replies)

Bug activity

Test Pilot bugs on Bugzilla

Latest checkins

Latest Checkins Feed

Initial blog post

Introducing Test Pilot

Test Pilot is an idea for a new user testing program for Mozilla Labs that aims to build a 1% representative sample of the Firefox user base for soliciting wide participation and structured feedback for Labs experiments.Test Pilot

Presently, whenever Mozilla publishes a new experiment we rely upon the blogosphere and word-of-mouth to attract the attention of people interested in trying it out. We then hope that some number of those early adopters will come back and find the appropriate place to provide feedback.

The problem is that we are systemically biasing feedback towards only those who happen to hear about a particular experiment (on a given day) and in those cases skew to the technically-savvy early adopter. It is therefore very hard to derive conclusions representative of our much larger and increasingly mainstream user base.

We also have no structured way in which to request feedback, and in particular, positive feedback. People are much more likely to take the time to dig in and figure out how to provide feedback if they have a negative reaction. It’s also likely true that the less technical adopters are confused when faced with Mozilla’s panoply of feedback options.

And finally, at the end of this “testing process,” we effectively throw away the interested people who have taken the time to volunteer as testers and start over again from scratch the next time around.

Overview of Test Pilot We can provide a much more satisfying experience all around by putting in place some basic infrastructure. Here’s the idea: 

   * We develop and promote a formal Test Pilot program with a Firefox add-on at its core.
   * The first time the Test Pilot add-on is run, it asks a few simple non-personally-identifiable questions in order to put the user into a demographic bucket, e.g. technical level, locale, etc., and to let them opt in to additional anonymous instrumentation.
   * Test Pilot will then notify its users when a new experiment is available for testing. If the user opts in, it will download the required software (if any) and load any information required to get started with the new experiment, e.g. overview, use cases, etc.
   * After either a specified amount of time or upon completion of a specific action, Test Pilot will prompt the user for feedback. The feedback form will only ask a few questions selected from a much larger set. A link will be provided to provided more comprehensive unstructured feedback or bug reports.
   * The set of questions posed for feedback will be randomly distributed within each demographic bucket to ensure statistical significance of the results.
   * Anonymized aggregate results and analysis will then be posted automatically to the Test Pilot site.
   * All participants will receive a “flight badge” displayed in their Test Pilot profile and available to embed on blogs, social networks, etc.

The idea is that by reducing the amount of required feedback to only a few clicks we can increase overall rates of participation.

While this will not take the place of our traditional testing and feedback mechanisms, Test Pilot should provide a new set of tools and capabilities to increase the efficiency and effectiveness of our user testing. It should also allow us to grow our community and mindshare with people who want to help but to whom we haven’t yet given the opportunity to participate.

Roadmap We’re pulling together a basic framework now with plans to launch the first version within the next few weeks. Specific features and roadmap are being developed in the Test Pilot discussion forum on the Mozilla Labs site.

Getting Involved 

   * Discuss, debate, and add to the design in the Labs discussion forum
   * Jump into #labs on irc.mozilla.org

Project blurb + tagline

Join to be a Test Pilot!

Test Pilot is a platform aimed at collecting structured user feedback from Firefox and other Mozilla Labs experiments. As a Test pilot, not only will you try before anyone else the newest features and coolest user-interface ideas, you could also learn how those test results may contribute to future designs. The goal for this platform is to encourage everyone from all skill levels to improve the Web experience by participating in these tests.

Once you install the Test Pilot add-on, you will be automatically signed into our tests. You will receive notifications on upcoming and finished tests. If the test requires you to install a new feature or product, the platform will ask for your permission.

When a product is not easy to use, don't just wonder why it's designed that way. Make your mark on the design and help us improve it! Join the forces, we want you!

User resources

Developer resources

List of "Developer Resources" to be linked (with URLs, if external to Labs site)

Top banner bullets (3)

Copy and links for three top level bullets, ie: "Where do I start?", "How does it work?", and "How can I help?" -- these can any three things you want.

What is a Test Pilot? A platform to collect structured user feedback from Firefox and other Mozilla Labs experiments.

How can I contribute? Join to be a Test pilot and share your feedback on product designs with the wider community.

Privacy policy All user feedback will be stored and published only in anonymous and aggregated form.

User resource pages

  • Why bother to be a Test pilot?

As a Test pilot, not only will you try before anyone else the newest features and coolest user-interface ideas, you could also learn how those test results may contribute to future designs.

  • How should I start to test?

Once you install the Test Pilot add-on, you will be automatically signed into our tests. You will receive notifications on upcoming and finished tests. If the test requires you to install a new feature or product, the platform will ask for your permission.

Developer resource pages

Copy for any internal "Developer Resources" pages to be included (Get involved, how to get started, etc)

Privacy + Legal notices

 This is a draft.

Mozilla Test Pilot Privacy Policy

Last Updated: July __, 2009

We’re working hard to protect your privacy while delivering products and services that bring you the performance and protection you desire in your personal computing. This Mozilla Test Pilot Privacy Policy (“Policy”) explains how Mozilla Corporation (“Mozilla”), a wholly-owned subsidiary of the non-profit Mozilla Foundation, collects and uses information from user tests conducted through Mozilla’s Test Pilot program (“Test Pilot”). It does not apply to other Mozilla websites, products or services, each of which have their own privacy policy. If you are using another Mozilla product at the time you are participating in Test Pilot, you should look at the relevant privacy policy to determine what other information is being collected. What is Test Pilot?

Test Pilot was created to provide a new set of tools and capabilities to increase the efficiency and effectiveness of our user testing. Test Pilot is a stand-alone platform. Test Pilot may be used as an add-on to your web browser, or bundled into other products. However Test Pilot is made available, you will always have to choose to participate in the program and unless you do so, no data will be collected by TestPilot. Please visit https://labs.mozilla.com/projects/test-pilot/ for more information on Test Pilot.

[QUESTION: I’d like to incorporate the Test Pilot Principles into the policy. I think they are god and would add to the policy. Thoughts?] Information Collected from Test Pilot Participants

In order to improve our user testing, Mozilla will collect certain data in relation to your participation in Test Pilot tests, which is in addition to data that may be collected normally from Firefox users. See Mozilla’s Firefox Privacy Policy for further information on data collected from all Firefox users.

You must provide opt-in consent to take part in any Test Pilot test. Before you opt in to a test, you will have an opportunity to review a description of the test, including its duration and the types of data that will be collected through the test (the “Test Description”). The data required for the test (“Test Data”) will be collected locally on your computer in a database within Firefox. At the end of the test, you will be able to review the Test Data and will be asked whether you would like to submit the Test Data to Mozilla.

The Test Data consist of information that cannot be directly associated with a specific person or entity, and include your technical level, computer configuration, the version of Firefox or other products that you use, the URL or Uniform Resource Locators (the addresses of web pages you navigate), or Internet Protocol (“IP”) addresses (the addresses of your computer on the internet). The Test Data do not include your name or email address, and will not be combined with any information that personally identifies you.

If you change your mind about participating in a test, you may opt out from the test at any time before you choose to share the Test Data, and the Test Data will not be transmitted to Mozilla. Similarly, if you choose not to submit the Test Data at the end of the test, the Test Data will not be transmitted to Mozilla. Transfer of Data to the U.S.

By registering for Test Pilot, you consent to the transfer of the Test Data and any other information collected under this Policy to the United States, which may provide a lesser level of data security than in your country of residence.[] Information Disclosure

Other than its wholly-owned subsidiaries listed below, Mozilla does not share the raw Test Data with any third party. We only provide the raw Test Data to Mozilla employees and contractors, as well as employees and contractors of the following wholly-owned subsidiaries of Mozilla: MZ Denmark ApS, MZ Canada Ltd, and MOZ 2008 Corporation. Mozilla requires its employees and contractors to sign confidentiality agreements that prohibit them from using the raw data other than for internal Mozilla purposes, or from disclosing these data to third parties.

All Test Data will be made anonymous and aggregated, and made available to the general public in the form of aggregate data. These aggregate data do not personally identify any test participants.

Test Pilot is designed to remove the Test Data from the local database in your Firefox browser when (1) you choose to opt out of Test Pilot or the test; (2) you choose to submit Test Data to Mozilla, or (3) you choose not to submit the Test Data when requested at the end of the test. Data Retention

We will retain Test Data for the period necessary to fulfill the purposes outlined in this Policy unless a longer retention period is required by law and/or regulations. Policy Changes

Mozilla may change this Policy from time to time. Any and all changes will be reflected on this page. When Mozilla changes this Policy in a material way, a notice will be posted on the www.mozilla.com Web site. Substantive changes may also be announced through the standard mechanisms by which Mozilla communicates with its users and community, including Mozilla's "announce" mailing list and newsgroup. It is your responsibility to ensure that you understand the terms of this Policy. You should periodically check this page for any changes to the current policy. Questions or Concerns

You may request appropriate access, correction, or deletion of Test Data, as permitted by law. We will seek to comply with such requests, provided that we have sufficient information to identify the Test Data related to you. Since the Test Pilot program collects very limited information about participants, it may not be possible to identify Test Data that you submitted. We recommend that you print a copy of the Test Data for your records before you submit it to Mozilla.

Any such requests or other questions or concerns regarding this Policy and Mozilla's data protection practices should be addressed to:

Mozilla Corporation Attn:Legal Notices - Privacy 650 Castro Street Suite 300 Mountain View, CA 94041-2021 Phone: +1-650-903-0800 E-mail: privacy@mozilla.com

We shall keep a copy of this Policy. If, at any time, you would like to see a copy of it, you should write to the above address or email address.

Retrieved from "https://wiki.mozilla.org/index.php?title=Labs/Site_2.0/Existing_site_replacement/Site_content/Test_Pilot&oldid=161718"