Generating a Trend Report (Quality Analytics)
Using Quality Analytics API, you can generate a report that shows coverage over time.
If you need to generate a trend report, you can reuse the following sample code to generate a report and get the URL to view it.
Please checkout SeaLights API Reference to see the full and latest API syntax available.
Script Dependencies
jq
curl
Script Configuration
Variable | Required | Type | Default | Description |
---|---|---|---|---|
DOMAIN | ✔ | string | ❌ | The domain for your SeaLights account lab. |
SL_API_TOKEN | ✔ | string | ❌ | An API token for your lab (generated in the settings tab of the web app) |
Script Output
The output of the script is written to:
report.url
https://<DOMAIN>/quality-analytics/trend-report/single/<REPORT_ID>;from_public_api=true
Generating a Coverage Trend Report
Coverage Trend report allows the user to select test stages from a single application and see aggregated data over time or over reference builds.
This report will show:
Selected test stages overall and modified coverage (aggregated, and broken down per test stage)
Entire build coverage and modified coverage (aggregated across all test stages, optional additional chart element)
Number of methods and modified methods (optional additional chart element)
More information about the Coverage Trend report can be found here: https://sealights.atlassian.net/wiki/spaces/SUP/pages/2832564225
Report Configuration
Variable | Required | Type | Default | Description |
---|---|---|---|---|
APP_NAME | ✔ | string | ❌ | App to report |
BRANCH_NAME | ✔ | string | ❌ | Branch to report |
ALL_BUILDS | ❌ | boolean | true | When true, show trend over time. |
INTERVAL | ❌ | ONE_WEEK | | ONE_MONTH | When ALL_BUILDS is true, defines at which intervals to calculate coverage. |
SELECTED_TEST_STAGES | ❌ | string[] | 8 most active test stages | Defines which test stages should be included in the report. |
DATE_RANGE_LABEL | ❌ | LAST_MONTH | | LAST_SIX_MONTHS | Defines the date range of the report |
REPORT_START_DATE | ❌ | number ms | ❌ | When date range is CUSTOM, defines the start date of the report. |
REPORT_END_DATE | ❌ | number ms | ❌ | When date range is CUSTOM, defines the end date of the report. |
NUM_OF_METHODS_CHART_ELEMENT | ❌ | boolean | false | the report will include an additional chart element representing the number of methods. |
ALL_TEST_STAGES | ❌ | boolean | false | the report will include an additional chart line representing all test stages coverage. |
COVERAGE_GATE_CHART_ELEMENT | ❌ | boolean | false | Additional chart line representing the coverage gate. |
Script
#!/usr/bin/env bash
DOMAIN="<domain>"
SL_API_TOKEN="<api-token>"
# Edit these variables to set the desired report configuration
APP_NAME="app1"
BRANCH_NAME="branch1"
#SELECTED_TEST_STAGES=("Unit Tests" "E2E Tests")
#DATE_RANGE_LABEL="LAST_SIX_MONTHS"
#REPORT_START_DATE="2022-01-22"
#REPORT_END_DATE="2022-02-22"
#ALL_BUILDS="true"
#INTERVAL="ONE_MONTH"
#NUM_OF_METHODS_CHART_ELEMENT="false"
#ALL_TEST_STAGES="false"
#COVERAGE_GATE_CHART_ELEMENT="false"
echo "Creating request payload"
if [ -z ${APP_NAME+x} ]; then
echo "Missing app name"
exit 1
fi
if [ -z ${BRANCH_NAME+x} ]; then
echo "Missing branch name"
exit 1
fi
REQUEST_PAYLOAD=$(
jq -n \
--arg appName "$APP_NAME" \
--arg branchName "$BRANCH_NAME" \
'{appName: $appName, branchName: $branchName}'
)
if [ ! -z ${ALL_BUILDS+x} ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg allBuilds "$ALL_BUILDS" \
'.allBuilds |= $allBuilds'
)
fi
if [ "$ALL_BUILDS" == "true" -a ! -z "${INTERVAL+x}" ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg interval "$INTERVAL" \
'.interval |= $interval'
)
fi
if [ ! -z "${SELECTED_TEST_STAGES+x}" ]; then
TEST_STAGES_ARR=$(for i in "${SELECTED_TEST_STAGES[@]}"; do echo "\"$i\""; done)
TEST_STAGES_JSON_ARR="[$(sed 's/" "/", "/g' <<< "$(echo $TEST_STAGES_ARR)")]"
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--argjson testStages "$TEST_STAGES_JSON_ARR" \
'.testStages |= $testStages'
)
fi
if [ ! -z "${DATE_RANGE_LABEL+x}" ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg dateRangeLabel "$DATE_RANGE_LABEL" \
'.dateRangeLabel |= $dateRangeLabel'
)
fi
if [ "$DATE_RANGE_LABEL" == "CUSTOM" ]; then
if [ -z ${REPORT_START_DATE+x} ]; then
echo "Missing start date"
exit 1
fi
if [ -z ${REPORT_END_DATE+x} ]; then
echo "Missing end date"
exit 1
fi
REPORT_START_DATE_UNIX=`date -d "${REPORT_START_DATE}" +%s%3N`
REPORT_END_DATE_UNIX=`date -d "${REPORT_END_DATE}" +%s%3N`
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--argjson from $REPORT_START_DATE_UNIX \
--argjson to $REPORT_END_DATE_UNIX \
'.from |= $from | .to |= $to'
)
fi
if [ ! -z ${DATE_RANGE_LABEL+x} ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg dateRangeLabel "$DATE_RANGE_LABEL" \
'.dateRangeLabel |= $dateRangeLabel'
)
fi
if [ ! -z ${NUM_OF_METHODS_CHART_ELEMENT+x} ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg numOfMethodsChartElement "$NUM_OF_METHODS_CHART_ELEMENT" \
'.numOfMethodsChartElement |= $numOfMethodsChartElement'
)
fi
if [ ! -z ${ALL_TEST_STAGES+x} ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg allTestStages "$ALL_TEST_STAGES" \
'.allTestStages |= $allTestStages'
)
fi
if [ ! -z ${COVERAGE_GATE_CHART_ELEMENT+x} ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg coverageGateChartElement "$COVERAGE_GATE_CHART_ELEMENT" \
'.coverageGateChartElement |= $coverageGateChartElement'
)
fi
echo "Getting report url: ${REQUEST_PAYLOAD}"
RESPONSE=$(curl -s -X POST -H "Authorization: Bearer $SL_API_TOKEN" \
-H "Accept: application/json" -H "Content-Type: application/json" --data "${REQUEST_PAYLOAD}" \
"https://$DOMAIN/sl-api/v1/quality-analytics/trend-reports/single/url"
)
ERROR_MSG=$(
echo $RESPONSE | jq -r '.error.message'
)
ERROR_CODE=$(
echo $RESPONSE | jq -r '.error.code'
)
REPORT_URL=$(
echo $RESPONSE | jq -r '.data.reportUrl'
)
if [ "${ERROR_CODE}" != "null" ]; then
echo $ERROR_MSG
exit $ERROR_CODE
fi
echo "Saving url '$REPORT_URL' to './report.url'"
echo $REPORT_URL > 'report.url'
Generating a Group Coverage Trend Report
Group Coverage Trend report allows the user to select multiple applications and see aggregated data over time.
This report will show:
All applications aggregated overall coverage and modified coverage
Per application overall coverage and modified coverage
Number of methods and modified methods (optional additional chart element)
More information about the Group Coverage Trend report can be found here: https://sealights.atlassian.net/wiki/spaces/SUP/pages/2832564282
Report Configuration
Variable | Required | Type | Default | Description |
---|---|---|---|---|
APP_BRANCHES | ✔ | AppBranch[] | ❌ | List of App-Branches for the report. |
INTERVAL | ❌ | ONE_WEEK | | ONE_MONTH | Defines at what intervals to calculate coverage. |
DATE_RANGE_LABEL | ❌ | LAST_MONTH | | LAST_SIX_MONTHS | Defines the date range of the report. |
REPORT_START_DATE | ❌ | number ms | ❌ | When date range is CUSTOM, defines the start date of the report. |
REPORT_END_DATE | ❌ | number ms | ❌ | When date range is CUSTOM, defines the end date of the report. |
NUM_OF_METHODS_CHART_ELEMENT | ❌ | boolean | false | the report will include an additional chart element representing the number of methods. |
Script
#!/usr/bin/env bash
DOMAIN="<domain>"
SL_API_TOKEN="<api-token>"
# Edit these variables to set the desired report configuration
APP_BRANCHES='[{"appName": "app1", "branchName": "branch1"}]'
#DATE_RANGE_LABEL="LAST_SIX_MONTHS"
#REPORT_START_DATE="2022-01-22"
#REPORT_END_DATE="2022-02-22"
#INTERVAL="ONE_MONTH"
#NUM_OF_METHODS_CHART_ELEMENT="false"
echo "Creating request payload"
if [ -z ${APP_BRANCHES+x} ]; then
echo "Missing app-branch list"
exit 1
fi
REQUEST_PAYLOAD=$(
jq -n \
--argjson appBranches "$APP_BRANCHES" \
'{appBranches: $appBranches}'
)
if [ ! -z "${INTERVAL+x}" ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg interval "$INTERVAL" \
'.interval |= $interval'
)
fi
if [ ! -z "${DATE_RANGE_LABEL+x}" ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg dateRangeLabel "$DATE_RANGE_LABEL" \
'.dateRangeLabel |= $dateRangeLabel'
)
fi
if [ "$DATE_RANGE_LABEL" == "CUSTOM" ]; then
if [ -z ${REPORT_START_DATE+x} ]; then
echo "Missing start date"
exit 1
fi
if [ -z ${REPORT_END_DATE+x} ]; then
echo "Missing end date"
exit 1
fi
REPORT_START_DATE_UNIX=`date -d "${REPORT_START_DATE}" +%s%3N`
REPORT_END_DATE_UNIX=`date -d "${REPORT_END_DATE}" +%s%3N`
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--argjson from $REPORT_START_DATE_UNIX \
--argjson to $REPORT_END_DATE_UNIX \
'.from |= $from | .to |= $to'
)
fi
if [ ! -z ${DATE_RANGE_LABEL+x} ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg dateRangeLabel "$DATE_RANGE_LABEL" \
'.dateRangeLabel |= $dateRangeLabel'
)
fi
if [ ! -z ${NUM_OF_METHODS_CHART_ELEMENT+x} ]; then
REQUEST_PAYLOAD=$(
echo $REQUEST_PAYLOAD | jq \
--arg numOfMethodsChartElement "$NUM_OF_METHODS_CHART_ELEMENT" \
'.numOfMethodsChartElement |= $numOfMethodsChartElement'
)
fi
echo "Getting report url: ${REQUEST_PAYLOAD}"
RESPONSE=$(curl -s -X POST -H "Authorization: Bearer $SL_API_TOKEN" \
-H "Accept: application/json" -H "Content-Type: application/json" --data "${REQUEST_PAYLOAD}" \
"https://$DOMAIN/sl-api/v1/quality-analytics/trend-reports/group/url"
)
ERROR_MSG=$(
echo $RESPONSE | jq -r '.error.message'
)
ERROR_CODE=$(
echo $RESPONSE | jq -r '.error.code'
)
REPORT_URL=$(
echo $RESPONSE | jq -r '.data.reportUrl'
)
if [ "${ERROR_CODE}" != "null" ]; then
echo $ERROR_MSG
exit $ERROR_CODE
fi
echo "Saving url '$REPORT_URL' to './report.url'"
echo $REPORT_URL > 'report.url'