Sonargraph offers a wide range of checks for different aspects of software quality: Architecture, metric thresholds, cyclic dependencies, duplicate code, etc.
When Sonargraph gets introduced for software projects that have been developed a couple of years, chances are high that hundred’s of issues are reported. But getting flooded with issues and sifting through them at regular intervals is neither efficient nor really helpful. Instead, you want to get a quick overview whether or not the quality is going into the right direction.
And this is where quality gates come into the game: they allow focussing on the information important to you.
This blog post details how this functionality is implemented in Sonargraph.
Motivation
Most developers and architects already know some code smells and metrics that they want to monitor and Sonargraph offers a large number issues and metrics that are calculated on different levels. If the issue or metric is not provided out of the box, the Script API allows implementing custom checks and metrics (check related blog posts like “Implementing Additional Metrics Using the Sonargraph Script API“).
Thresholds can be defined for each metric, so that issues are created and you can see errors or warnings for elements that violate those thresholds.
But, what if you have 132 threshold violations?
You want to ensure that this number is lower in the next release and that existing violations don’t get worse. Same for other types of issues like cycle groups: you want to gradually improve the current state, i.e. you want to ensure that no new cycle groups are introduced and that existing ones get smaller.
Quality gates allow the definition of higher-level conditions. It should be possible to define conditions not only for the current state but also based on changes with respect to a baseline. Samples for quality gate conditions:
- “Less than 10 architecture violations”.
- “No threshold violations for metric ‘TotalLines’ at level ‘Source File’.”
- “No additional error issues.”
- “An increase in the Average Component Dependency (ACD) must be lower than 5%.”
- “Number of architecture violations must be reduced by at least 10%.”
How is this implemented in Sonargraph?
Functionality
In Sonargraph, a quality gate consists of conditions that can either be defined on the current system state or based on the trend with respect to a baseline. The baseline and trend info is provided by the ‘System Diff’ analyzer, that has been described in “Using Sonargraph’s “System Diff” for Continuous Code Quality Improvements“.
Conditions for the current system state are focused on issues and allow the definition of an upper limit to their count, e.g. “Less than 10 Architecture Violations”. If you want to define a condition on a metric value, you need to first configure a metric threshold and then define a limit for this violation as shown in the following screenshot (click to enlarge):
If the “System Diff” is activated and a baseline exists, conditions can also be defined on the development of metric values. This is useful, because values of coupling metrics like Average Component Dependency (ACD) are growing even for well structured systems. You want to monitor them, but you are rather interested in “spikes”, i.e. sudden increases in their value as this is usually an indicator for a newly introduced problem. A quality gate condition can be defined for exactly this purpose and you can specify a threshold for an absolute value diff and/or a relative value diff.
The threshold values can even be negative, so a condition can represent a goal to be reached, e.g. “Change of metric value for architecture violations must be <= -10%”, see screenshot below:
The “Quality Gates” view indicates in the upper half, which conditions pass. Issues and metrics that match a condition are listed in the lower half.
Advanced Issue Tracking
The ‘System Diff’ provides information about added, modified and removed issues. For threshold violations, cycle groups and duplicate code blocks, it also highlights the trend, i.e. “Have more lines been added to an already huge file?” or “Have more elements been added to an existing cycle group?” and of course also the opposite if existing issues have been improved.
Side Note
The merging of cycle groups as well as a split of an existing cycle group into smaller groups are also detected.
This algorithm to match issues from the baseline with issues of the current state has also been transferred to the resolution matching of Sonargraph: definitions of “Ignore” and “Fix” resolutions for cycle group and duplicate code block issues now tolerate changes in the involved elements. Additional details about the matching functionality can be found in the user manual, chapter 13 “Examining Changes“.
Conditions involving threshold violations, cycle groups or duplicate code blocks support two types of checks: “Strict” and “Relaxed”. The relaxed check tolerates, if an existing issue is slightly getting worse: A worsened threshold violation is tolerated if no threshold is defined for the value change or if the change is below the defined threshold. Cycle group issues are tolerated, if only the number of parser dependencies to remove has been increased. And similarly, if only the number of duplicated lines of existing duplicate code blocks has been increased, but the number of involved occurrences stayed the same.
More details can be found in the Sonargraph user manual, chapter 14 “Defining Quality Gates“.
Recommended Usage
Sonargraph supports any number of quality gates, so that related conditions can be grouped together. Similar to architectures and scripts, only “checked” quality gates are validated. Failed quality gates produce issues which appear in the Sonargraph “Issues” view. The “Quality Gates” view provides details about the issues and metric values that caused a condition to fail.
If you use SonargraphBuild for a headless check on your CI server, you can activate your core quality gates and let the build fail if quality gates don’t pass. This is achieved by specifying a failset on quality gate issues (more details can be found in the documentation of SonargraphBuild):
<sonargraphBuild
...
<failSet failOnEmptyWorkspace="true">
<include issueType="QualityGateIssue" />
</failSet>
</sonargraphBuild>
Info about quality gates is contained in the HTML report as well as reported on the command line:
Checking active quality gate(s)...
[Failed] Quality Gate 'Structure.xml'
[Passed] Current System Conditions:
[Passed] Condition "<= 10 issues of type 'Core:ArchitectureViolation' with severity
'Error' and resolution 'None'" [5 issues matched (0 excluded)]
[Passed] Condition "<= 0 threshold violations for metric 'Core:SourceFile:TotalLines'
with severity 'Error, Warning' and resolution 'None'"
[0 issues matched (0 excluded)]
[Failed] Baseline Conditions:
[Passed] Condition "No additional or worsened issues of type
'Core:NamespaceCycleGroup' with severity 'Error, Warning' and resolution
'None', using 'Relaxed' check" [0 issues matched (0 excluded, 0 tolerated)]
[Failed] Condition "Change of metric value for
'Core:System:ViolationsParserDependencies' must be <= -10,00%"
[5 -> 5 (+0,00%)]
Check of quality gates failed.
Quality Gate Summary: 1 of 1 failed.More specific checks can be selectively activated in Sonargraph, e.g. to provide some guidance during review sessions.
Summary
This blog post outlined the functionality of quality gates to monitor the current state and trends of quality indicators. They allow the definition of custom quality goals and focussing on those metrics and issue types that you find the most important.
Quality gate conditions that monitor the trend in metric values and issues are especially useful if Sonargraph is introduced in existing projects, as they help to gradually improve the quality of your code base.