Integrating Test Executions from various Testing Frameworks
You can integrate any Testing Framework with Sealights using our Public API to capture Tests Events like Start and End of Test Stage, Tests results, etc.
We’re listing below some examples of integrations for your reference.
The sample code listed in this article is not supported by Sealights as an official plugin, agent, or product and is shared as a courtesy.
Sealights Integration samples repository
Sealights had published a repository of several sample integrations on Github:
Known Third Party Integrations
Robot Framework Sealights Custom Listener
This section demonstrates applying SeaLights with Test Optimization to tests executed with the Robot framework (Python): It uses a reference implementation of the Robot Listener interface to facilitate calls to SeaLights API at appropriate phases of the test suite's lifecycle. The SeaLights integration is implemented in the file SLListener.py available as an open-source project for you to be able to customize it to your specific configuration.
By default, the command line to add the Sealights' listener to the Robot execution is as follows: --listener “SLListener.py:<CustomerDomain>:<Token>:<buildSessionId>:<Test Stage Name>:<LabId>”.
The command arguments are specified after the listener name (or path) using a colon (:) as a separator:
Customer Domain URL
Token
buildSessionId
Test Stage name
(Optional) LabId
For example:
robot --listener "SLListener.py:your-domain.sealights.co:`cat sltoken.txt`:`cat buildSessionId.txt`:Robot Tests" some_tests.robotor
robot --listener "SLListener.py:your-domain.sealights.co:${SL_TOKEN}:${SL_BUILD_SESSION_ID}:Robot Tests:${SL_LAB_ID}" some_tests.robotKatalon Studio Sealights plugin
This plugin is publicly available on Katalon Store.
Ginger by Amdocs
Since Release 4.5, Ginger by Amdocs has the ability to integrate with Sealights using a built-in plugin and allows to:
Publish Automation execution data to Sealights, to deliver Faster, High-Quality, and controlled releases
Get Test Recommendations from Sealights for Ginger Runset execution to optimize the efforts and reduce testing cycle time
Sample Code
Converting Results Report from XML
Several frameworks provide the ability to report the results of test executions into an XML format that can be reused to feed Sealights.
For example, the Windows Powershell commands below allow to convert a TestNG XML format into the JSON format expected by Sealights API.
If your Testing Frameworks uses TestNG, please prefer Sealights' Out-Of-the-Box integration with TestNG. If your framework simply outputs its result in this format, the sample below is relevant for you.
$SealightsJson = ConvertTo-Json -InputObject @( [xml]$(Get-Content -Path .\testng-results.xml) | Select-Xml -XPath "//test-method" |foreach {
[PSCustomObject]@{
name = $_.node.signature
start = ([DateTimeOffset](Get-Date $($_.node.'started-at'))).ToUnixTimeMilliseconds()
end = ([DateTimeOffset](Get-Date $($_.node.'finished-at'))).ToUnixTimeMilliseconds()
status = $(if ($_.node.status -eq 'PASS') { 'passed' } else { $(if ($_.node.status -eq 'FAIL') { 'failed' } else { 'skipped' }) })
}
} ) )If you’re converting an NUnit XML report, you’ll be mapping test-case object and its following properties: fullname, start-time, end-time, and result (with values Passed, Failed, and Skipped)
Handling Unexpected Closure of Testing Executions
In case, your testing execution is aborted because of any failure, it is important to close all the executions left open because the Testing framework was not able to perform a graceful shutdown for example.
Below is a sample code that checks the list of remaining open executions and closes them one by one as part of a post-build section in a Jenkins pipeline script used to execute the tests.