...
The TIA can identify and execute the smallest subset of mandatory tests, without compromising quality. It is good for both automated tests and manual tests.
Who can use the TIA & how?
The TIA can be used by any SeaLights customer, for all types of tests and for any app and programming language supported by SeaLights.
In order to user use the TIA, you must meet the following conditions:
Test events must be reported to SeaLights (either automatically or manually by using SeaLights Chrome Extension)
Your Test Runner (or Manual Tester) can get a list of tests to be skipped from the coming test cycle.
Or, you can use SeaLights UI for manual testing recommendations.
There are two ways to automatically report tests and connect to your Test Runner:
...
Integrate SeaLights with your CI.Status colour Purple title you
Install agents, then direct SeaLights to scan the latest build and the tests that are already running.Status colour Purple title you
Correlate each test with the methods that relate to it.Status colour Yellow title SeaLights
Compare any new build with the build which the current test stage was last executed.Status colour Yellow title SeaLights
Identify any all tests that are impacted by modified content, and recommend tests.Status colour Yellow title SeaLights
Execute only a subset of your tests, based on the recommended tests to skip.Status colour Purple title you
Execute "Full Run" test cycle on defined scheduled to ensure no tests are ever overlooked.Status colour Purple title you
Analyze and improve the code-to-test correlation by using machine-learning algorithms.Status colour Yellow title SeaLights
...
The page displays the actual test execution time per app, whether TIA is ON or not.
This view should help you easily detect long execution times and understand where the main problems are.
...
Each line is clickable and opens a more detailed view for each test stage.
Turning The Simulator ON
Users You can use the TIA Simulator to understand the impact of the TIA on the test execution time.
...
Clicking the button switches all TIA pages to simulation mode and a blue bar joins the Test Execution bar, to make it easy to compare the time difference. The blue bar represent the potential total execution time of any app / test stage / build (depends on the view you are looking at).
...
The default configuration is that the TIA is OFF for all apps. You can turn it ON anytime on an app level or for specific branch. It is done per test stage.
Where and how you turn the TIA ON (or OFF if needed)?
When you select app and then test stage on Test Optimization page, you go to a page where you can see the Yes/No button.
...
The button if off by default:
...
TIA ON for Automated Tests
...
TIA is now available for manual tests reported by the Chrome extension or APIs.
TBD
Full Run Configuration
TBD.
There are two options to consume the TIA recommendations:
On SeaLights UI, go to the relevant app and test stage page and select the build you are about to test.
The recommendations list will be available automatically, if the TIA is ON.Use SeaLights API to get the list of tests to skip.
Full Run Configuration
TIA builds the code-to-test connection by learning which test hit which area in the code. The more hits the TIA captures per test, the more accurate the connection map. Therefore, it is recommended to have a full run with all the tests every one in a while, to increase the accuracy of the TIA.
How should you know what the frequency of a full run should be?
For test stage you run a few times a day, it makes sense to set the full run policy to once a day.
For test stage you run a few times a week, it makes sense to set the full run policy to once a week.
You can select the option to run it every X times you run the test stage (X is customisable).
Notice! After working with the TIA ON for a while, if you identify the savings are not as you expected, consider changing the full run policy to be less frequent.
When can you set the full run policy?
A Settings link is available next to the TIA ON button, once you turn on the TIA ON button.
...