Group Coverage Trend Report
- Nurit Grinstein (Unlicensed)
This feature is available only to customers who have provided consent to the usage of SnowFlake database. To learn more please contact Customer Success. |
|---|
The Group Coverage Trend Report is the second report type available under Quality Analytics. It displays multiple apps code coverage over time.
Generating A Report
From the Quality Analytics main page / reports list, click on the “Create new report” button. Generating a report is as simple as selecting the apps and branches you are interested in. Each app has its own color that is used in the charts.
You have different options to view the data:
Date Range
The popular options available are: Last month, Last 2 months, Last 3 months, Last 6 months, Last year.
If none of the above is what you need, you can always select a custom date range. We recommend a minimum of 4 weeks.
Please note that the start date for the data shown in the charts depends on the selected interval (see section below) and is currently limited to the following Monday or the 1st of the month.
Interval
You can select in which intervals you would like to aggregate the data. Options are: 1 week, 2 weeks, 3 weeks, 4 weeks, 1 month. When selecting 1 month, the data shown in the chart will start at the 1st of the relevant month. For the rest of the interval options, the data shown in the chart will start at the following Monday.
Understanding The Data
The report includes two charts: Modified Coverage and Overall Coverage.
Both charts display an aggregated coverage on top of the specific apps coverage and include:
Y-axis: Coverage % (0-100%)
X-axis: Time, according to the filter selection and intervals
Intervals
When selecting any of the interval options (1 week, 2 weeks, 3 weeks, 4 weeks, or 1 month) any point in the X-axis and in the chart line represents an interval. Interval X takes all code changes from the last build in interval X-1, and takes all coverage from all related test stages that were reported in all builds within interval X. (e.g. Interval of 1 month: coverage in July means all code changes since last build on June and their coverage from all test stages executions reported on builds within July).
Modified Coverage Chart
Modified coverage is the coverage of your code changes. Why is it important to track it? Your quality risks are code changes you haven’t tested. The higher the coverage, the lower the chances for escaping defects. You want to see high coverage overtime, to reduce risks. High modified coverage overtime is an indication for a good quality culture. It means that any new or modified code is tested.
Overall Coverage Chart
Overall coverage is the coverage of your entire code. You should track the overall coverage to make sure that overtime your trend is positive, mainly by covering new and modified code. You might see drops in cases you had major code changes with no coverage.
Advanced Report Features
Drill-Down To A Specific App Coverage Trend Report
When reviewing the data included in a group trend report, you might need to drill down to a certain app level in order to examine which test stages contributed to its coverage. In order to quickly view the wanted report click on the “Open report” button from the right legend panel.
Number Of Methods
When examining your group coverage trend, seeing the data in the right context is important. To that end we have added an option to visualize the scope of the change in the modified chart and the scope of the apps that are examined in the overall chart. To add the methods bar to your charts, click on the “Chart elements” and toggle on the “Number of methods” option.
Number Of Production Defects
One of the most common KPIs around quality is the number of production defects that were opened against a build or a version. SeaLights now allows you to visualize the number of production defects and compare it to the coverage trend. To add the number of production defects as added bars in your charts, click on the “Chart elements” and toggle on the “Number of production defects” option. You will be required to upload a .CSV file containing the data of the relevant production defects:
The file should contain relevant production defects to the apps and the date range of the report.
Every line in the file should represent a production defect.
There should be a column in a UTC format that includes the date of the defect and is named “Date”.
Once the file is uploaded successfully, the data will appear on the charts. Please note:
The data will be added to both modified and overall coverage charts, as an aggregated bar (no breakdown according to the apps included in the report).