TIA for Manual Tests

Owned by Sarit Ariel
Last updated: Aug 22, 2022 by Nurit Grinstein
7 min read

Manual tests are supported throughout all of SeaLights' products, but while our coverage and TGA reports do not need to detect each particular manual test in order to show the full coverage and risk analytics, for the TIA to work properly, the system does need to learn about each of the manual tests separately.

Read more about the TIA tool to find out how it optimizes your test runs and saves time and resources.

Test detection

The first step in allowing the TIA tool to optimize manual tests is to allow the SeaLights engine to detect and learn about each separate manual test. There are three ways to achieve this:

  1. Using the SeaLights chrome extension to report manual tests

  2. Using the SeaLights manual tests wizard that is part of the Test Optimization page

  3. Using the SeaLights API to report the manual tests from any testing platform

 

Option 1 - Use the Chrome Extension “Manual Tests”

Installing and launching the SeaLights chrome extension

  1. Go to your Chrome store and search for “SeaLights Chrome extension”

  2. Download the extension to your local computer. You will need to do this for all computers used by your manual testers.

  3. If the extension is not automatically pinned to your bar, it is advised to pin it.

  4. Copy the extension token from the SeaLights UI by (a) going to the setting page, and then (b) under “integrations” click on “browser extension token

  5. Copy the token by clicking on the “copy token to clipboard”

  6. Open the extension setting (a) and paste the token (b)

 

Running tests using the SeaLights Chrome extension

The Chrome extension is designed to be used in parallel to the performance of the manual test.

While the manual tester is performing the test, they should also enter relevant information about the test to the extension.

There are three ways to ‘let the extension know’ which test is being performed by the tester:

  1. Enter a single test name to the extension each time

  2. Upload tests from a CSV file

  3. Use the tests recommended by the TIA engine for the build that is being tested

In any of the options, once the test is performed, it is registered by the system in the same way and is used for the production of recommendations for the next build in the branch

 

Entering a single test name to the extension each time

  1. Click on the SeaLights Chrome extension icon and on the “Start manual test” line.

  2. The default setting is to report the tests according to App / Branch / Build. Make sure the information there is correct to the test being performed.

  3. Choose the radio button “single test”

  4. Enter the test stage, the test suite, and the test name. If the test was previously run and reported to SeaLights, an auto-complete will help you find it quickly.

 

5. Click on “Save”. This will minimize the extension as follows:

Uploading a list of tests from a CSV file

  1. Click on the SeaLights Chrome extension icon and on the “Start manual test” line.

  2. The default setting is to report the tests according to App / Branch / Build. Make sure the information there is correct to the test being performed.

  3. Choose the radio button “test list” and than click on “Upload from file”

  4. The file should be of a .csv format: first column should include a header and then a list of test names, each test name in a separate row. Example file is attached below.

  5. After choosing a file and uploading it, you should see the list of all tests that the file contains. You are now able to:

    1. Unclick a test that you do not wish to execute this time, by unselecting it’s checkbox

    2. Enter a new one by clicking on the plus sign on the top of the page

    3. Change the tests order by grabbing one and dragging it

 

5. Once the list of tests in the extension represents the tests as you plan to perform them, click on ‘close’

6. At this point, you should begin the tests. Click on ‘start’

 

Use the test recommended by the TIA engine

  1. Click on the SeaLights Chrome extension icon and on the “Start manual test” line.

  2. The default setting is to report the tests according to App / Branch / Build. Make sure the information there is correct to the test being performed.

  3. Choose the radio button “test list” and than click on “Get recommendations”. This will automatically pull the list of recommended tests for the app / Branch / Build combination that you entered in the upper part of the extension

  4. Now, as before, you are now able to:

    1. Unclick a test that you do not wish to execute this time, by unselecting it’s checkbox

    2. Enter a new one by clicking on the plus sign on the top of the page

    3. Change the tests order by grabbing one and dragging it

5. Once the list of tests in the extension represents the tests as you plan to perform them, click on ‘close’

6. At this point, you should begin the test. Click on ‘start’

 

Running the tests

  1. After the click on start, the interface of the extension will minimize and will only show the name of the test that is being performed.

2. Click on the stop button in order to finish the execution of the test, and enter the result

3. Once the result is entered click on “Submit and done” to save the test and enter the next one

4. If you are performing a list of tests, click on “Submit and start next test”

 

Option 2 - Use the “Manual Tests” wizard (UI)

Another option to report Manual Tests is to use the Manual Tests wizard under the “Test Optimization” page.

  1. Go to the “Test Optimization” tab

  2. Click on the “New Manual Tests” button

The manual test wizard works exactly as described above for the extension, with only a few UI differences, which do not affect the overall usability of the product.

Manual tests in the SeaLights UI

  1. The tests reported by the extension will be shown in the dashboard under the relevant app, build, and branch

  2. In the Test Optimization page, the test will again be visible under the relevant branch and test stage

 

Test Recommendations

The recommended tests are available per build in the Test Optimization page, on the relevant app - test stage page.

  1. To see the recommended tests per build, make sure the relevant app and test stage has the TIA ON on the Test Optimization page.

    Or, if you are first interested in evaluating the recommendations, you can use the simulation mode and simply click the 'Run TIA Simulator' button.

  2. Click through to the relevant build, then click on the “Recommendations - All”. Notice that you can see recommendations for past builds as well as new builds that you haven’t tested yet.

  3. Recommended tests can be downloaded to a CSV file using the download button on the right top.

  4. Use the recommended tests list to execute only the necessary tests (+ any new tests you need to add). This will help you reduce your manual test cycle and still maintain a high level of quality.