CloudServices/Chronicle/Testing Notes
Test coverage
- Unit Tests: yes
- Smoke Tests: need
- Regression Tests:
- Integration Tests
- e2e Tests
- functional testing in firefox, uses theintern. need to see how we can use or add to this
- Load and Performance Tests: neither of these two (load/performance) are likely to be an issue yet, but could be looked at later
How are deployments done?
will follow the standard jenkins deploy pipeline by end of q1
commit into branch -> PR -> code review -> travis build -> lint it manually -> run it -> merge
master is currently deployed manually to staging, will be formalized soon by benson
merge to master will trigger a build in travis which will then deploy to stage/jenkins
ops will do staging first and then production later
2 week sprints, deploy at the end of each, promoted to staging
Does the project automatically execute the required test types on checkin via a build server like jenkins or travis? yes, it will when the jenkins work is complete
Can/should each of the test types fail the build on failed runs? unit and e2e should fail, load is unknown until required
Are the test results visible in treeherder or another tool? just in travis/jenkins, will need to standardize
Dev build should run: unit, functional tests, eventually a smoke/small load test
Stage build should run: unit, functional tests, eventually some additional e2e or api tests and a full load test
Next Steps
- Make sure unit and e2e/functional tests execute and can fail the build on jenkins
- Expand e2e tests as needed to cover all features
- Create a small load test to run in dev for every PR in travis (as much load as you can put on in ~5 minute max)
- Create any scripts or tools that would ease the dev->stage->prod process (any repetitive qa steps), best if they can run in jenkins. automated linting is one example.
- Create a full load test that can be run in stage in jenkins (where do we think the load will be in 6 months?)
