Settings functionality encompasses the following elements:

You should use Settings in the following cases:

• Create various views for the data based on code tagging/labeling.

• Control the data you’d like scope in or out from SeaLights Analysis

• Generate/copy/remove token for agent setup to be used automatically or manually or for API integration setup.

• Set up the app/branch pairs to be shown in the Test Gap Analytics report.

• Configure reports and notifications.

• Hide applications from the dashboard.

Settings Screen

To get to the Settings screen click on the settings wheel icon on the top right area of your screen:

You should see the following tabs on the left:

Tokens Management

a. For each token created you can Refresh/Copy/Download/Remove - you should choose either copy or download when you wish to use it in the various setups

b Agent Tokens - you need this token for agent setup as part of setting up your agent (manually or automatically) for scanning builds using the SeaLights Build Scanner or for listening to test executions while using SeaLights Test Listener. The start and end of the token is shown for ease of identification

c. API Tokens - you need this token for setting up an API integration with SeaLights system to SeaLights analyzed data through the Public API

d. Browser Extension Tokens - you need this token for setting up the Sealights Browser extension as indicated below:

• Select the SeaLights Chrome Extension icon on the browser bar

• Select the 3 dots icon to configure the Browser Extension token

• Paste the Chrome Extension token in the box provided

Data Management

Hidden Apps

Choose the Application you wish to Show/Hide (what is visible across the SeaLights dashboard)

Or Click on the App to choose which branches you wish to Show/Hide:

Code Scope (Ignore / Include)

The Code Scope page allows you to create, view and manage rules per app and test stage that define the code scope for the analysis of new builds reported to SeaLights. By using Ignore/Include rules, you can define which scope of the code should be analyzed in the Dashboard, Quality Analytics and TGA reports. For example, you might want to include the entire app’s code in the analysis, except for 1 folder which is a legacy code that you are not interested in getting coverage for. You can do so by using an Exclude rule.

• Scope rules will affect the analytics starting from the next build that will be reported to SeaLights.

The scope rules will not affect Test Optimization’s recommendations. Since SeaLights is obligated to improve your current product quality, Test Optimization withholds the highest standard for test recommendations. For this reason, and although other parts of the product allow setting your code scope, Test Optimization provides recommendations of impacted tests for your entire code base, including parts of the code that you set to be ignored in Settings.

• The rules work at the class level or file level, not at the method level (except for some special code entities - see more info below under “Global Code Entities Rule”).

Why do SeaLights use the class level or file level, and not at the method level?

We have found from our experience that the gain in accuracy in setting code scope at the method level is small whereas the increase in difficulty of maintaining it and performing coverage investigation over time is major. Therefore we concentrate on the class and file level, except for some special code entities we do support today. If you do still wish to ignore/include specific methods, we suggest refactoring your code to include them in their own file and ignoring / including it.

What can I do with the code scope capability?

The Code Scope page gives you the tools to:

• Search for apps and view the app’s rules, including when they were created and by who, and when they were last modified and by who

• Add new rules to include/ignore code

• Edit, duplicate or delete existing rules

You can define multiple rules per app and test stage (there is no importance to the rules order). In case of a conflict between an Include rule and an Ignore rule in the same scope, the Include rule should run over the Ignore rule, and the rest of the files will be ignored. For example, if the code files stored as followed:

• folder_A:

  • folder_B:

    • file1.ts

    • file2.ts

  • folder_C:

    • file3.ts

The files that will be included in the analysis as a result of the conflict between Ignore rule and Include rule are:

By applying an Include rule on a folder and then adding new files to the same folder, the new files will be included in the analytics. For example, in case of applying an Include rule on folder B and then adding a new file "file4.ts" to this folder, the new file "file4.ts" will be included as well as "file1.ts" and "file2.ts".

Selecting an App

You can navigate to a specific app by typing the app name and clicking on the relevant result.

By clicking on a specific app, a list of test stages associated with file rules is displayed. Each test stage from the list unfolds into a list of the relevant file rules (per the app and the test stage), by clicking on the test stage. For example, in case there are rules for the “Manual” test stage:

Global Code Entities Rule

Each app has a “Global code entities rule”, which automatically ignores some code entities according to the programming language of the app. For example, It might ignore Auto-Generated Code, Getters and Setters, constructors, etc.

You can change this rule by checking the code entities which you like to exclude from the analysis, and unchecking the code entities which you would like to include.

You can view when this rule was last modified and by who, by clicking on the arrow at the bottom of this field:

Add New Rule

You can add new rules per app and per test stage, to change your code scope. To set a new rule - click on “ADD NEW RULE”. After applying a new rule, you can quickly add new similar rules to the same app by duplicating an existing rule - click on “Duplicate rule”.

In case that no rules were added to this app:

In case that rules already have been added to the app:

After clicking on the “ADD NEW RULE” button, a new window will pop-up:

1. Test Stage field

The file rule is set per test stage, or for all test stages.

Select the test stages which the code files will be included/excluded from.

2. Rule Type field

Select if the relevant code should be excluded/included in the Dashboard, Quality Analytics and TGA reports.

Including code will analyze only the selected code areas for Quality Risks and Coverage. You might want to use the include rule to better fit the analysis for a small scope of the code, and to avoid irrelevant code areas.

Ignoring code will remove it from being analyzed for Quality Risks and Coverage. You might want to exclude code that is either auto-generated, getters, setters, default constructors, and other irrelevant code areas such as third party or code you knowingly choose to exclude from the analysis.

3. Rule Condition field and Path field

The rules define which code files will be in the scope, by matching between a condition and a path. In the path field, insert the text that will be matched with the rule condition. You can use the auto-correct recommendations to avoid typos.

The match will allow you to filter code files by several options such as file name, file type, etc. For example, if there is a code file under the path - “sl-browser-agent/lib/agent-factory.ts”, using the following conditions:

1. Exact match - select a specific file (require the full logical path + filename). Example:  path: “sl-browser-agent/lib/agent-factory.ts” will select this specific file.

2. Starts with - select a folder/class and all of its content (note, do not use reg expressions syntax). Example: path: “lib” will select all the files within sl-browser-agent/lib/

3. Ends with - Select a file type or common file names. Example: path: “spec.ts” will select all the files which end with “*spec.ts”. "Agent-factory.ts" will not be selected using this rule.

4. Contains - a wildcard select - select all files and classes/folders which contain the entered string. Example: path: “factory” will select all the files whose path contains “factory”- like “agent-factory.ts”.

5. Entire app - select the entire app - Example: “sl-browser-agent”.

4. Reason field

You can document the reason why the selected code areas should be included/excluded from the selected test stages.

Edit, Duplicate or Delete Rule

Edit Rule

To edit an existing rule - click on “Edit rule”:

Duplicate Rule

To duplicate an existing rule - click on “Duplicate rule”. You may want to use this option in cases such as applying a rule on several test stages but not all of the test stages. The duplicated rule will be applied for the same app only.

Delete Rule

To delete an existing rule - click on “Delete rule”. The rule deletion will take effect starting from the next build reported to SeaLights.

The Code Scope table displays per application the app name, number of rules (sum of rules from all test stages), last modified date, and editors list (contains every user that added, edited or deleted a rule).

You can filter the table by app names search:

You can sort the table by the number of rules and by the date it was last modified (determined by the last app rule modification), in increasing/decreasing order:

Extra information about the rule including the rule’s reason (if given), creation date and by who, last modified date and by who, will be displayed by clicking on the rule:

Code Labels

SeaLights allows you to divide your code into areas of interest by defining categories representing either teams (Development, QA, etc), Functional Areas (Login, Reports, etc) or any division you’d like to see within your codebase. Furthermore, under each category, you can define labels that will represent the code area relevant to that label. When using the SeaLights dashboard you can filter your view according to these categories and labels defined here. You can also define labels as “High Priority” so Quality Risks around that label will be highlighted.

How to define Categories/Labels:

1. Click on "Create New Label Rule"

2. Add a Label Category as highlighted below - Simply provide a Category name

3. Under a Category, you can add a new label by clicking on “+” and provide a label name

   

4. Now you should define the rules that will encapsulate the content you wish to tag/label, click on the label you wish to define on the left, and then pick the relevant code by choosing the app, how to filter it and the path for the code. 

5. Filtering the code, by adding rules:

   1. Exact match - Label a specific file (require the full logical path + filename)

      • Example:  sl-browser-agent/lib/agent-factory.ts

   2. Starts with - Label a folder/class and all of its content

      • Example: typing lib will label the highlighted area below → sl-browser-agent/lib/*.*

   3. Ends with - Label a file type or common file names

      • Example: typing spec.ts will label all the files which end with spec.ts

   4. Contains - a wildcard labeling - label all files and classes/folders where its name contains the entered string

   5. Entire app - Label the Entire app

6. Finally, you should decide whether this label represents a “High Priority” code. Once tagged as High Priority - this will be indicated in “red dot” in the dashboard per new quality risks that were found in that high priority code label:

   

How to Modify Existing Categories/Labels:

1. Search for the label which rules you want to edit, for example "Quality - QualityDocs":

   1. 

2. Click the edit icon

3. Go to the rule you want to edit and click the edit icon

4. Update the condition

5. And click the save button

Test Gaps Report Settings

Use these settings to define the Test Gaps Analytics reports - setting the reports here will result in reports that will be available in the Test Gaps Analytics screen.

Adding an application to the Test Gap Analytics 

1. Select the relevant app

2. Select the relevant branch

3. Select the start date (a future date - e.g. next Sprint start date or the 1st of the next Month)

4. Select the reporting period in weeks (Scheduled refresh)

5. Click "Save"

Removing an app from the Test Gap Analytics view can be done by clicking on the delete icon

Notifications

The notifications settings will enable you to push quality notifications to a selected list of attendees.

Notifications are highly effective once combined with proper Quality Gates - Get notified only once your Quality Gates are failing.

How to configure your notification policy?

1. Select the name of the application

2. Select the relevant branch

3. Choose when to be notified

   1. Builds that did not meet the Quality Gates

   2. Builds with Quality Risks

   3. Every completed build

4. Choose how to be notified by typing in the email address of the relevant attendees (you may send a notification to the Author/Contributor who was working on the code)

   1. Slack

   2. Email

TIA Settings

Quality Gates 

The Quality Gates administration can be found in the "Settings" area and will be permitted to user-admins only.

The Quality Gates screen presents all of your reported applications and will enable to set/modify the Quality gate for All apps / Specific app.

The new functionality helps in setting your quality policy and select the relevant apps that will be part of the policy scope.

• Edit Default - Enables to modify the default quality gates

  • Note - The default value will be applied on new applications only

• Apply Default to All - Apply the customer default on all apps (customer default is defined in "Edit Default" section)

• Edit a specific app Quality Gate - Click on the pencil icon on the right

Each Quality Cate consists of 4 parameters:

• Rule

  • New/Modified Code Coverage

  • Overall Code Coverage

  • Failed Tests

• Test Stages

  • Combined Across Stages

  • Specific Test Stage (e.g. Unit Tests, Integration Tests...)

• Operator (Fixed) - Less than / Greater than

• Value - Your quality threshold for the specific rule

Related articles