Generating a Trend Report (Quality Analytics)

Using Quality Analytics API, you can generate a report that shows coverage over time.
If you need to generate a trend report, you can reuse the following sample code to generate a report and get the URL to view it.

Please checkout SeaLights API Reference to see the full and latest API syntax available.

Script Dependencies

  • jq

  • curl

Script Configuration

Variable

Required

Type

Default

Description

Variable

Required

Type

Default

Description

DOMAIN

string

The domain for your SeaLights account lab.

SL_API_TOKEN

string

An API token for your lab (generated in the settings tab of the web app)

Script Output

  • The output of the script is written to: report.url

https://<DOMAIN>/quality-analytics/trend-report/single/<REPORT_ID>;from_public_api=true

Generating a Coverage Trend Report

Coverage Trend report allows the user to select test stages from a single application and see aggregated data over time or over reference builds.

This report will show:

  • Selected test stages overall and modified coverage (aggregated, and broken down per test stage)

  • Entire build coverage and modified coverage (aggregated across all test stages, optional additional chart element)

  • Number of methods and modified methods (optional additional chart element)

More information about the Coverage Trend report can be found here: https://sealights.atlassian.net/wiki/spaces/SUP/pages/2832564225

Report Configuration

Variable

Required

Type

Default

Description

Variable

Required

Type

Default

Description

APP_NAME

string

App to report

BRANCH_NAME

string

Branch to report

ALL_BUILDS

boolean

true

When true, show trend over time.
When false, show trend over reference builds.

INTERVAL

ONE_WEEK |
TWO_WEEKS |
THREE_WEEKS |
FOUR_WEEKS |
ONE_MONTH

ONE_MONTH

When ALL_BUILDS is true, defines at which intervals to calculate coverage.

SELECTED_TEST_STAGES

string[]

8 most active test stages

Defines which test stages should be included in the report.

DATE_RANGE_LABEL

LAST_MONTH |
LAST_TWO_MONTHS |
LAST_THREE_MONTHS |
LAST_SIX_MONTHS |
LAST_YEAR |
CUSTOM

LAST_SIX_MONTHS

Defines the date range of the report

REPORT_START_DATE

number ms

When date range is CUSTOM, defines the start date of the report.

REPORT_END_DATE

number ms

When date range is CUSTOM, defines the end date of the report.

NUM_OF_METHODS_CHART_ELEMENT

boolean

false

the report will include an additional chart element representing the number of methods.

ALL_TEST_STAGES

boolean

false

the report will include an additional chart line representing all test stages coverage.

COVERAGE_GATE_CHART_ELEMENT

boolean

false

Additional chart line representing the coverage gate.

Script

#!/usr/bin/env bash DOMAIN="<domain>" SL_API_TOKEN="<api-token>" # Edit these variables to set the desired report configuration APP_NAME="app1" BRANCH_NAME="branch1" #SELECTED_TEST_STAGES=("Unit Tests" "E2E Tests") #DATE_RANGE_LABEL="LAST_SIX_MONTHS" #REPORT_START_DATE="2022-01-22" #REPORT_END_DATE="2022-02-22" #ALL_BUILDS="true" #INTERVAL="ONE_MONTH" #NUM_OF_METHODS_CHART_ELEMENT="false" #ALL_TEST_STAGES="false" #COVERAGE_GATE_CHART_ELEMENT="false" echo "Creating request payload" if [ -z ${APP_NAME+x} ]; then echo "Missing app name" exit 1 fi if [ -z ${BRANCH_NAME+x} ]; then echo "Missing branch name" exit 1 fi REQUEST_PAYLOAD=$( jq -n \ --arg appName "$APP_NAME" \ --arg branchName "$BRANCH_NAME" \ '{appName: $appName, branchName: $branchName}' ) if [ ! -z ${ALL_BUILDS+x} ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg allBuilds "$ALL_BUILDS" \ '.allBuilds |= $allBuilds' ) fi if [ "$ALL_BUILDS" == "true" -a ! -z "${INTERVAL+x}" ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg interval "$INTERVAL" \ '.interval |= $interval' ) fi if [ ! -z "${SELECTED_TEST_STAGES+x}" ]; then TEST_STAGES_ARR=$(for i in "${SELECTED_TEST_STAGES[@]}"; do echo "\"$i\""; done) TEST_STAGES_JSON_ARR="[$(sed 's/" "/", "/g' <<< "$(echo $TEST_STAGES_ARR)")]" REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --argjson testStages "$TEST_STAGES_JSON_ARR" \ '.testStages |= $testStages' ) fi if [ ! -z "${DATE_RANGE_LABEL+x}" ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg dateRangeLabel "$DATE_RANGE_LABEL" \ '.dateRangeLabel |= $dateRangeLabel' ) fi if [ "$DATE_RANGE_LABEL" == "CUSTOM" ]; then if [ -z ${REPORT_START_DATE+x} ]; then echo "Missing start date" exit 1 fi if [ -z ${REPORT_END_DATE+x} ]; then echo "Missing end date" exit 1 fi REPORT_START_DATE_UNIX=`date -d "${REPORT_START_DATE}" +%s%3N` REPORT_END_DATE_UNIX=`date -d "${REPORT_END_DATE}" +%s%3N` REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --argjson from $REPORT_START_DATE_UNIX \ --argjson to $REPORT_END_DATE_UNIX \ '.from |= $from | .to |= $to' ) fi if [ ! -z ${DATE_RANGE_LABEL+x} ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg dateRangeLabel "$DATE_RANGE_LABEL" \ '.dateRangeLabel |= $dateRangeLabel' ) fi if [ ! -z ${NUM_OF_METHODS_CHART_ELEMENT+x} ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg numOfMethodsChartElement "$NUM_OF_METHODS_CHART_ELEMENT" \ '.numOfMethodsChartElement |= $numOfMethodsChartElement' ) fi if [ ! -z ${ALL_TEST_STAGES+x} ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg allTestStages "$ALL_TEST_STAGES" \ '.allTestStages |= $allTestStages' ) fi if [ ! -z ${COVERAGE_GATE_CHART_ELEMENT+x} ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg coverageGateChartElement "$COVERAGE_GATE_CHART_ELEMENT" \ '.coverageGateChartElement |= $coverageGateChartElement' ) fi echo "Getting report url: ${REQUEST_PAYLOAD}" RESPONSE=$(curl -s -X POST -H "Authorization: Bearer $SL_API_TOKEN" \ -H "Accept: application/json" -H "Content-Type: application/json" --data "${REQUEST_PAYLOAD}" \ "https://$DOMAIN/sl-api/v1/quality-analytics/trend-reports/single/url" ) ERROR_MSG=$( echo $RESPONSE | jq -r '.error.message' ) ERROR_CODE=$( echo $RESPONSE | jq -r '.error.code' ) REPORT_URL=$( echo $RESPONSE | jq -r '.data.reportUrl' ) if [ "${ERROR_CODE}" != "null" ]; then echo $ERROR_MSG exit $ERROR_CODE fi echo "Saving url '$REPORT_URL' to './report.url'" echo $REPORT_URL > 'report.url'

Generating a Group Coverage Trend Report

Group Coverage Trend report allows the user to select multiple applications and see aggregated data over time.

This report will show:

  • All applications aggregated overall coverage and modified coverage

  • Per application overall coverage and modified coverage

  • Number of methods and modified methods (optional additional chart element)

 

More information about the Group Coverage Trend report can be found here: https://sealights.atlassian.net/wiki/spaces/SUP/pages/2832564282

Report Configuration

Variable

Required

Type

Default

Description

Variable

Required

Type

Default

Description

APP_BRANCHES

AppBranch[]

List of App-Branches for the report.

INTERVAL

ONE_WEEK |
TWO_WEEKS |
THREE_WEEKS |
FOUR_WEEKS |
ONE_MONTH

ONE_MONTH

Defines at what intervals to calculate coverage.

DATE_RANGE_LABEL

LAST_MONTH |
LAST_TWO_MONTHS |
LAST_THREE_MONTHS |
LAST_SIX_MONTHS |
LAST_YEAR |
CUSTOM

LAST_SIX_MONTHS

Defines the date range of the report.

REPORT_START_DATE

number ms

When date range is CUSTOM, defines the start date of the report.

REPORT_END_DATE

number ms

When date range is CUSTOM, defines the end date of the report.

NUM_OF_METHODS_CHART_ELEMENT

boolean

false

the report will include an additional chart element representing the number of methods.

Script

#!/usr/bin/env bash DOMAIN="<domain>" SL_API_TOKEN="<api-token>" # Edit these variables to set the desired report configuration APP_BRANCHES='[{"appName": "app1", "branchName": "branch1"}]' #DATE_RANGE_LABEL="LAST_SIX_MONTHS" #REPORT_START_DATE="2022-01-22" #REPORT_END_DATE="2022-02-22" #INTERVAL="ONE_MONTH" #NUM_OF_METHODS_CHART_ELEMENT="false" echo "Creating request payload" if [ -z ${APP_BRANCHES+x} ]; then echo "Missing app-branch list" exit 1 fi REQUEST_PAYLOAD=$( jq -n \ --argjson appBranches "$APP_BRANCHES" \ '{appBranches: $appBranches}' ) if [ ! -z "${INTERVAL+x}" ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg interval "$INTERVAL" \ '.interval |= $interval' ) fi if [ ! -z "${DATE_RANGE_LABEL+x}" ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg dateRangeLabel "$DATE_RANGE_LABEL" \ '.dateRangeLabel |= $dateRangeLabel' ) fi if [ "$DATE_RANGE_LABEL" == "CUSTOM" ]; then if [ -z ${REPORT_START_DATE+x} ]; then echo "Missing start date" exit 1 fi if [ -z ${REPORT_END_DATE+x} ]; then echo "Missing end date" exit 1 fi REPORT_START_DATE_UNIX=`date -d "${REPORT_START_DATE}" +%s%3N` REPORT_END_DATE_UNIX=`date -d "${REPORT_END_DATE}" +%s%3N` REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --argjson from $REPORT_START_DATE_UNIX \ --argjson to $REPORT_END_DATE_UNIX \ '.from |= $from | .to |= $to' ) fi if [ ! -z ${DATE_RANGE_LABEL+x} ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg dateRangeLabel "$DATE_RANGE_LABEL" \ '.dateRangeLabel |= $dateRangeLabel' ) fi if [ ! -z ${NUM_OF_METHODS_CHART_ELEMENT+x} ]; then REQUEST_PAYLOAD=$( echo $REQUEST_PAYLOAD | jq \ --arg numOfMethodsChartElement "$NUM_OF_METHODS_CHART_ELEMENT" \ '.numOfMethodsChartElement |= $numOfMethodsChartElement' ) fi echo "Getting report url: ${REQUEST_PAYLOAD}" RESPONSE=$(curl -s -X POST -H "Authorization: Bearer $SL_API_TOKEN" \ -H "Accept: application/json" -H "Content-Type: application/json" --data "${REQUEST_PAYLOAD}" \ "https://$DOMAIN/sl-api/v1/quality-analytics/trend-reports/group/url" ) ERROR_MSG=$( echo $RESPONSE | jq -r '.error.message' ) ERROR_CODE=$( echo $RESPONSE | jq -r '.error.code' ) REPORT_URL=$( echo $RESPONSE | jq -r '.data.reportUrl' ) if [ "${ERROR_CODE}" != "null" ]; then echo $ERROR_MSG exit $ERROR_CODE fi echo "Saving url '$REPORT_URL' to './report.url'" echo $REPORT_URL > 'report.url'