Document Overview

Timeline:

Architecture

In this section, the product's architecture is described. Any individual components or actors are identified, their "knowledge" or what data they store is identified, and data flow between components and external entities is described.

The main objective of this feature/product is: This is a test case management system. It is Free and open-source. This is our upcoming replacement for Litmus. The system is designed to provide test case management, and status on test execution. Test managers can write tests for testers to execute, and they can see the results of the tests in real-time.

We will host an instance of this product as our own test case management system. However, the source is checked into github here and other organizations will deploy their own instances.

Use Cases

Roles and functions

There are 4 main roles. These are the roles and main use-cases.

• Testers - Testers can only execute tests. In this case, usually community members will be given a URL to an existing Test Run.

• Test Creators - These users can do everything that Testers do, but they also can create new tests. These are "semi-deputized" members of the community. They will create new tests, and a Test Manager or Admin will activate them, if they pass muster.

• Test Managers - These are usually in-house QA or Product owners, or fully deputized Community members. These users can specify new Products, and Product Versions that should be tested. They specify the environments that must be tested for those products. They can create new Test Cases and Suites of Test Cases and activate them so they can be run. They can also create Test Runs (collections of Test Suites that can be executed). Once this use activates the Test Run, then all the users above can execute the test cases and provide results.

• Admins - These users can do everything from above, and can also manipulate user accounts.

Running Tests

All users can run tests. They nav to a test run, specify the environment they're using (such as Windows 7) and begin executing test cases. They mark them Pass / Fail / or Invalid (if the case is unclear to the point they can't tell if it passes or fails).

Viewing Results

All users can login and see results of executed test cases. They can see what passed, what didn't and what cases were not clear. Users can also see % coverage for test runs. Users can also "drill down" and see if all the environments for a specific test case of a run have been executed.

Manage

This is where the user changes the items in the system. Management is specific per role (as outlined above).

History

Originally this product was designed to be in two parts. A "Platform" written in Java providing REST endpoint APIs and a "User Interface" written in Django. Despite some people's investment in this relationship with uTest and the separation of the two parts, eventually it was determined that this architecture was untenable. At this point (Nov 2011) we decided to re-architect as pure Django.

One result of this change, is that the product is no longer a joint project with uTest, however they own the name "Case Conductor." So we will be changing the name. The branding search is underway.

Documentation: http://readthedocs.org/docs/case-conductor/en/latest/index.html

Project Plan:Pivotal Tracker

Components

Describe any major components in the system and how they interact. Also include any third-party APIs (those Mozilla does not control) and what type of data is sent or received via those APIs.

Deployment Considerations: readthedocs

The main components of the product are:

• Web App
• BrowserID (external component)
• File Store / Attachments (external component)

Web App

This component manages:

• Products & Versions
• Environments
• Test Runs, Suites & Cases
• Test Results
• Users / Authentication

The user can choose to login with either browserID, or they can just have a username, email and password stored with us. Email and username are stored as strings in the db. Passwords are salted and hashed with bcrypt, using the django-sha2 app built by mozillian Frederic Wenzel.

The tables below simply summarize the data encountered by this component.

Stored Data:

Communication with File-Store / Attachments Attachments are files stored in a file store. In our deployment, it's just the local file system. Full documentation on the Django file store API can be found here: Django File Store API

Communication with BrowserID Documentation on using BrowserID with a web app can be found here: BrowserID Dev Guide

User Data Risk Minimization

In this section, the privacy champion will identify areas of user data risk and recommendations for minimizing the risk.

Breach and Log Disclosure

The Risk is that information logged by the Web App server will be unintentionally disclosed to untrusted third parties through a breach event.

Recommendation: Work with infrasec and IT/Ops to minimize logging, minimize retention window, and deploy a secure logging infrastructure.

Alignment with Privacy Operating Principles

In this section, the privacy champion will identify how the feature lines up with Mozilla's privacy operating principles.

See Also: Privacy/Roadmap_2011#Operating_Principles:

Principle: Transparency / No Surprises

Users of the system must create an account and use the system specifically for testing. The system's operation must be clear to the users, both by opening the source (done) and by providing information to users at appropriate times about data collection, etc. For users who upload test cases (things they created themselves), it's obvious and unsurprising what happens.

The Risk: is that test-performing users may be submitting data to the Web App without knowing they're submitting it. For example, I may run a test and submit my results but it may not be clear whether the results contain my browsing history, search queries, etc. It should be clear when I submit data what I'm submitting.

Recommendations: Ensure that users submitting test data to the system are aware of the types of data being submitted (and have access to the data itself). This can be done through a prompt ("Hey, thanks for finishing the test, we're going to collect x, y, z from you now") or a dashboard where users can see the type of data they submit.

Principle: Real Choice

Users must be able to participate in the system without being asked for unreasonably wide data sets, and they must understand how the system interacts with their data. It's pretty clear that all users of this system intend to help with creating, performing or analyzing tests.

Recommendations: None.

Principle: Sensible Defaults

Users of our other products should not be required to use this system (they're not).

Recommendations: None.

Principle: Limited Data

Data collected by the system should be limited to what is required for the intended functionality.

The Risk is that we may end up with data that never gets used for the clear value proposed to users of the system.

Recommendations: Work with infrasec and IT/Ops to minimize logging, minimize retention window, and deploy a secure logging infrastructure.

Follow-up Tasks and tracking