Auto-tools/Projects/web-platform-tests
Contents
Overview / Problem
web-platform-tests is a testsuite designed for cross-browser testing of web features like CSS and DOM APIs.
Creating, and running, such a testsuite is a crucial part of improving the open web. Whenever a technology has multiple implementations it's very easy to introduce accidental discrepancies that cause poor interoperability. Historically this has been a problem on the web with different browsers all having their own quirks that developers have had to learn and work around. By contributing to the web-platform-tests testsuite we can build up a repository of test cases that can be run by all browser vendors through their development cycle so such issues are caught before they make it into released builds.
Goals & Considerations
The aim of this project is to allow us to run the full set of web-platform-tests in Gecko and Servo, and create an easy workflow for contributing tests back upstream.
Non Goals
We are not aiming to own the entire set of tests or push our full set of tools to all other browser vendors.
Dependencies / Who will use this
The target here is Mozilla developers and release management. We run tests on every change to the browser and the feedback about passing/failing tests is important. Also updated/new tests need to run upstream so expect other vendors to run these tests.
Design & Approach
Milestones / Future Work
- Improved support for upstreaming tests from gecko directly into the upstream repository.
- Support reliable testing in automation for debug builds.
- Allow running CSS tests using wptrunner.
- More sophisticated reftest matching for cases where deterministic pixel equality isn't achievable.
Implementation
The overall web-platform-tests project is hosted on github and documented at testthewebforward.org. This site contains notes on how to run and write tests.
The tool we use for running the tests is wptrunner, which is documented on readthedocs.
The tests use a custom web server, wptserve, which also has documentation on readthedocs.
Gecko-specific integration lives in-tree and is documented in some README files.
Getting Involved
Because a lot of the project is developed "upstream" on GitHub, not all of the bugs are in bugzilla.mozilla.org.
Good First Bugs
These tend to be labeled with difficulty:easy on GitHub:
