You can integrate any Testing Framework with Sealights using our Public API to capture Tests Events like Start and End of Test Stage, Tests results, etc.
We’re listing below some examples of integrations for your reference.

1 Sealights Integration samples repository
2 Known Third Party Integrations
3 Sample Code
- 3.1 Converting Results Report from XML
- 3.2 Handling Unexpected Closure of Testing Executions

The sample code listed in this article is not supported by Sealights as an official plugin, agent, or product and is shared as a courtesy.

Sealights Integration samples repository

Sealights had published a repository of several sample integrations on Github:

https://github.com/Sealights/sealights-integration-examples

Known Third Party Integrations

Robot Framework Sealights Custom Listener

This section demonstrates applying SeaLights with Test Optimization to tests executed with the Robot framework (Python): It uses a reference implementation of the Robot Listener interface to facilitate calls to SeaLights API at appropriate phases of the test suite's lifecycle. The SeaLights integration is implemented in the file SLListener.py available as an open-source project for you to be able to customize it to your specific configuration.

By default, the command line to add the Sealights' listener to the Robot execution is as follows: --listener “SLListener.py:<CustomerDomain>:<Token>:<buildSessionId>:<Test Stage Name>:<LabId>”.

The command arguments are specified after the listener name (or path) using a colon (:) as a separator:

Customer Domain URL
Token
buildSessionId
Test Stage name
(Optional) LabId

For example:

robot --listener "SLListener.py:your-domain.sealights.co:`cat sltoken.txt`:`cat buildSessionId.txt`:Robot Tests" some_tests.robot

or

robot --listener "SLListener.py:your-domain.sealights.co:${SL_TOKEN}:${SL_BUILD_SESSION_ID}:Robot Tests:${SL_LAB_ID}" some_tests.robot

For more information about Robot Framework Interfaces, please refer to the official documentation: https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#listener-interface

Katalon Studio Sealights plugin

This plugin is publicly available on Katalon Store.

Ginger by Amdocs

Since Release 4.5, Ginger by Amdocs has the ability to integrate with Sealights using a built-in plugin and allows to:

Publish Automation execution data to Sealights, to deliver Faster, High-Quality, and controlled releases
Get Test Recommendations from Sealights for Ginger Runset execution to optimize the efforts and reduce testing cycle time

Sample Code

Converting Results Report from XML

Several frameworks provide the ability to report the results of test executions into an XML format that can be reused to feed Sealights.

For example, the Windows Powershell commands below allow to convert a TestNG XML format into the JSON format expected by Sealights API.

If your Testing Frameworks uses TestNG, please prefer Sealights' Out-Of-the-Box integration with TestNG. If your framework simply outputs its result in this format, the sample below is relevant for you.

$SealightsJson = ConvertTo-Json -InputObject @( [xml]$(Get-Content -Path .\testng-results.xml) | Select-Xml -XPath "//test-method" |foreach {
    [PSCustomObject]@{
        name = $_.node.signature
		start = ([DateTimeOffset](Get-Date $($_.node.'started-at'))).ToUnixTimeMilliseconds()
		end = ([DateTimeOffset](Get-Date $($_.node.'finished-at'))).ToUnixTimeMilliseconds()
		status = $(if ($_.node.status -eq 'PASS') { 'passed' } else { $(if ($_.node.status -eq 'FAIL') { 'failed' } else { 'skipped' }) })
    }
} ) )

If you’re converting an NUnit XML report, you’ll be mapping test-case object and its following properties: fullname, start-time, end-time, and result (with values Passed, Failed, and Skipped)

For better accuracy of Sealights Test Optimization, please make sure to set your XML report timestamps in milliseconds.
For example, you can change the parameter timestampFormat in your XML Reporter settings from yyyy-MM-dd'T'HH:mm:ss'Z' to yyyy-MM-dd'T'HH:mm:ss.SSS'Z', for that purpose.

Handling Unexpected Closure of Testing Executions

In case, your testing execution is aborted because of any failure, it is important to close all the executions left open because the Testing framework was not able to perform a graceful shutdown for example.

Below is a sample code that checks the list of remaining open executions and closes them one by one as part of a post-build section in a Jenkins pipeline script used to execute the tests.

Details on the Get Executions Status List API can be found in our dedicated documentation page Test Sessions API (a.k.a TIA API) | Get Executions Status List.
For example, you have the ability to query the list of executions with a filter based on the Test Stage name or the LabId in addition to the execution status “created”.

    post {
        aborted{
            echo '[Sealights] Cleanup executions left open.'
            withCredentials([string(credentialsId: 'sl.agent.token', variable: 'SL_TOKEN')]) {
                sh '''
                    set +x
                    SL_SESSION_IDs=(`curl -sX GET "https://$DOMAIN/sl-api/v1/executions?bsid=$BSID&status=created" \
                                              -H "Authorization: Bearer $SL_TOKEN" -H "Content-Type: application/json" \
                    						  | jq -r '.data.list | map(.executionId) | join(" ")'`)
                    #Optional: filter based on the Test Stage name and/or labId (especially relevant in case of parallel testing)
                    
                    echo "Found ${#SL_SESSION_IDs[@]} executions"
                    
                    for id in ${SL_SESSION_IDs[@]}
                    do 
                    	echo -n "Closing Test Session ID $id: "
                    	curl -isX DELETE "https://$DOMAIN/sl-api/v1/test-sessions/$id" \
                    	  -H "Authorization: Bearer $SL_TOKEN" -H "Content-Type: application/json" | grep HTTP
                    done
                '''
            }
        }
    }

Support

Integrating Test Executions from various Testing Frameworks

Analytics