DRAFT: Sealights and Parallel testing within a Test Stage (1 Lab) use case
Ref doc: https://sealights.atlassian.net/wiki/spaces/DEV/pages/18251777
Why do this?
Running parallel test cases (automated and manual) against the same environment (lab). This is an anti-pattern not recommended by sealights that has downsides (listed in red below)
How to do this?
1. Use Chrome Extension for Manual Testing using the exact same test stage as the automated tests. However, if the text isn't exactly the same, a separate test stage would be created
2. Kickoff Manual Testing as a test stage via script using the same test stage as the automated tests
Alternatives / Recommended Pattern
Have an environment with its own labID dedicated to automated testing.
Comparison
Compare n test stages per lab vs 1 test stage per lab
| Parallel Testing (1 Lab) | Serial / Isolation (1 stage: 1 lab) |
|---|---|---|
TIA support (out of box) | Not supported | Supported |
TIA support (process change) | Supported (w/ clean serial run) | n/a |
Process change required | Yes | Yes |
Additional Infra required | No | Yes |
Important Notes:
First test stage start that Sealights detects starts the test stage, last test stage end recieved will end the test stage
Coverage will be for all the test cases executed during the time the test stage is open. The downside of this is the coverage report presented by sealights won't be able to distinguish which test suite was responsible for coverage.
In general if testing stages use the same environment at the same time then they must use the same test stage name and the coverage presented for that test stage would represent coverage of all testing done during that test stage.
In order to get recommendations for a particular test stage / test type, make sure that periodically (every week / two for example) you are able to run just that test stage, without running other test stages in parallel.