Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The TIA can identify and execute the smallest subset of mandatory tests, without compromising quality. It is good for both automated tests and manual tests.

Who can use the TIA & how?

The TIA can be used by any SeaLights customer, for all types of tests and for any app and programming language supported by SeaLights.

In order to user use the TIA, you must meet the following conditions:

  • Test events must be reported to SeaLights (either automatically or manually by using SeaLights Chrome Extension)

  • Your Test Runner (or Manual Tester) can get a list of tests to be skipped from the coming test cycle.
    Or, you can use SeaLights UI for manual testing recommendations.

There are two ways to automatically report tests and connect to your Test Runner:

...

  1. Status
    colourPurple
    titleyou
    Integrate SeaLights with your CI.

  2. Status
    colourPurple
    titleyou
    Install agents, then direct SeaLights to scan the latest build and the tests that are already running.

  3. Status
    colourYellow
    titleSeaLights
    Correlate each test with the methods that relate to it.

  4. Status
    colourYellow
    titleSeaLights
    Compare any new build with the build which the current test stage was last executed.

  5. Status
    colourYellow
    titleSeaLights
    Identify any all tests that are impacted by modified content, and recommend tests.

  6. Status
    colourPurple
    titleyou
    Execute only a subset of your tests, based on the recommended tests to skip.

  7. Status
    colourPurple
    titleyou
    Execute "Full Run" test cycle on defined scheduled to ensure no tests are ever overlooked.

  8. Status
    colourYellow
    titleSeaLights
    Analyze and improve the code-to-test correlation by using machine-learning algorithms.

...

The page displays the actual test execution time per app, whether TIA is ON or not.
This view should help you easily detect long execution times and understand where the main problems are.

...

Each line is clickable and opens a more detailed view for each test stage.

Turning The Simulator ON

Users You can use the TIA Simulator to understand the impact of the TIA on the test execution time.

...

Clicking the button switches all TIA pages to simulation mode and a blue bar joins the Test Execution bar, to make it easy to compare the time difference. The blue bar represent the potential total execution time of any app / test stage / build (depends on the view you are looking at).

...

The default configuration is that the TIA is OFF for all apps. You can turn it ON anytime on an app level or for specific branch. It is done per test stage.

Where and how you turn the TIA ON (or OFF if needed)?

When you select app and then test stage on Test Optimization page, you go to a page where you can see the Yes/No button.

...

The button if off by default:

...

TIA ON for Automated Tests

...

TIA is now available for manual tests reported by the Chrome extension or APIs.

TBD

Full Run Configuration

TBD.

There are two options to consume the TIA recommendations:

  • On SeaLights UI, go to the relevant app and test stage page and select the build you are about to test.
    The recommendations list will be available automatically, if the TIA is ON.

  • Use SeaLights API to get the list of tests to skip.

Full Run Configuration

TIA builds the code-to-test connection by learning which test hit which area in the code. The more hits the TIA captures per test, the more accurate the connection map. Therefore, it is recommended to have a full run with all the tests every one in a while, to increase the accuracy of the TIA.

How should you know what the frequency of a full run should be?

  • For test stage you run a few times a day, it makes sense to set the full run policy to once a day.

  • For test stage you run a few times a week, it makes sense to set the full run policy to once a week.

  • You can select the option to run it every X times you run the test stage (X is customisable).

Notice! After working with the TIA ON for a while, if you identify the savings are not as you expected, consider changing the full run policy to be less frequent.

When can you set the full run policy?

A Settings link is available next to the TIA ON button, once you turn on the TIA ON button.

...