Overview of the TIA Process

Owned by John Vanderzyden (Unlicensed)
Last updated: Feb 06, 2020
3 min read

The outcome of the TIA is a recommended test list which consists of 4 tests types:

  1. Impacted tests -Tests related to recent code changes 

  2. Failed Tests -Tests that didn’t pass in the last run

  3. Pinned/Flagged Test -Tests marked as important to run in every test cycle

  4. New Tests -Tests that have never been run before

How does it work?

  1. Integrate with your CI

  2. Scan the latest build and the tests that are running

  3. Correlate each test with the methods related to it

  4. Compare new build content with content in which the current test stage was last executed

  5. Identify tests impacted by changed content and recommend tests

  6. Execute only the recommended tests, excluding those not “relevant”

  7. Execute "Full Run" tests on defined scheduled (first of day/week/month or N builds) to ensure no tests are ever overlooked

  8. Analyze and improve using machine-learning algorithms



Product Walkthrough



Click on TIA on the top right end of the screen



The default configuration is that automatic test selection is "Off" (The TIA analysis will provide a list of test recommendations without executing the recommended test list).



Pick the application name and the relevant branch

Choose the date range of the TIA analysis

Select the test stage to be analyzed





Understanding the Data

The high-level report includes 4 main levels of information

  1. Summary view 

    • Includes all of your applications sorted by the total time saved (Per app, Per Test Stage)

  2. Application details

    • includes app name, branch name, analysis period, and the relevant test stage

  3. Summary

    • aggregated data such as

    1. Average run-time and Average # of tests (with or without tests selection)

    2. Average time saved and Average # of tests reduced (with or without tests selection)

    3. Estimated total time saved for the analysis period

  4. Test Execution details

    • per build information

    1. Build number & Build date

    2. Tests Recommended

    3. Estimated Run Time







Test Recommendations Report

This report presents a list of impacted tests per build (also available via SeaLights APIs)