B2G/QA/API Permissions Test Plan
Contents
Boot2Gecko API Permissions Test Plan
Summary
| Leads | David Chan (irc: dchan) Geo Mealer (irc: geo) |
| Contributors | TBD |
| References | Firefox OS Security Model Overview |
Scope
This test plan covers Gecko permissions interaction from the point an app has installed into the permissions database downwards. It does not cover installation flow of the app itself, aside from verifying that the permissions manager properly reflects installed permissions.
Strategy
The original testing methodology for this was a simple round trip: we would install an app with a given set of permissions, then check that the app behaved consistently with those permissions. However, we hit a number of problems where testing dependencies were either unstable or not fully implemented and would interfere with one of more legs of this multi-step process.
This test plan takes a more isolated approach to testing permissions interaction, so that AUT instability doesn't block the entire test approach. As such, it's split into three major sections: what permissions look like by default, whether a (non-default) permission setting is properly honored by the API, and whether installed apps properly set permissions in the manager.
These test suites will likely be implemented in mochitest, with a possible dependency on mochitest-chrome for the third suite to allow completion of install flow.
Suites
Suite 1: Verify default permissions
Given an app context, and without modifying its permissions, are the default permissions as expected?
Tracking Bug
Methodology
- Install or host an app with a context of either content, hosted, installed, or certified.
- Don't include a custom manifest or otherwise modify the permissions table.
- Query permissions manager directly for app and check against expected values.
Dependencies
- Install flow
- Permissions manager
Suite 2: Verify API behavior
Given permissions in the database, does API allow/deny/prompt as expected?
Tracking Bug
Methodology
- Start from the context with lowest permissions (content, probably).
- Add permissions directly via SpecialPowers
- Test API allows/denies/prompts correctly.
Dependencies
- Individual APIs
- Permissions Manager
Suite 3: Verify permissions database population
Given an app context and manifest, does the permissions database populate correctly?
Tracking Bug
Methodology
- Install (or host) application with customized manifest
- Query permissions database directly for app and check against expected values.
Dependencies
- Installation flow
- Permissions Manager
Design Notes
In combination, these suites give similar assurance as round trip, but have distinct advantages on a dependency front. The biggest dependency becomes the permissions manager. Even with APIs broken we get verification of default permissions and population, and even with install flow broken we get verification of APIs.
The biggest downside of this approach is that APIs that check the app context directly (as opposed to honoring the permissions manager) will cause problems. If the API blocks the content context unconditionally, it can't be tested via Suite 2. If it passes some other context unconditionally, we wouldn't catch it here.
Augmenting Suite 2 by testing from each of the different contexts should catch each of those scenarios, but reintroduces a dependency on install flow in that suite. Once the code base has stabilized, this would be a worthy expansion to the tests.
