ConQAT – The Continuous Quality Assessment Toolkit

Benjamin Hummel, @BenjaminHummel, CQSE GmbH, http://www.softwarequalityblog.eu/

Long-lived software systems are subject to gradual quality decay if no counter measures are taken. Continuous quality control counters this decay through an orchestrated application of constructive and analytic quality assurance techniques. The Continuous Quality Assessment Toolkit ConQAT provides the tool-support required enacting continuous quality control in practice. It supports the rapid design and configuration of quality dashboards that integrate diverse quality analysis methods and tools. The dashboards come with a set of interactive tools that support the in-depth inspection of identified quality defects and help to prevent the introduction of further deficiencies. To cope with the diversity of real-world software systems, ConQAT is not limited to the analysis of source code but takes into account various types of other development artifacts like models or textual specifications. Through its flexible architecture, ConQAT can be customized to address the quality requirements that are truly relevant for a software project. Thereby, it helps to successfully counter quality decay of software systems.

Web Site: http://www.conqat.org/
Version tested: 2015.2
License & Pricing: Apache Public License 2.0
Support: Provided by CQSE GmbH (http://www.cqse.eu/)

Design Principles of ConQAT

ConQAT was initially created by the Group for Software Quality and Maintenance at the Technical University of Munich in 2005. Starting from the early days, it was created around a set of design principles that guide its development until these days.

Quality data needs aggregation and visualization

Automatically analyzing the quality of large software systems generates an enormous amount of data. To prevent users being overwhelmed by too much data, ConQAT aggregates analysis results so that they are comprehensible and presents them in an appropriate manner. For this, ConQAT must be aware of the analyzed system’s structure to produce meaningful results. As even highly aggregated results must be effectively conveyed to users, ConQAT provides powerful visualization mechanisms that go beyond tables and charts.

Provide dedicated views

Assessment results must be accessible to all project stakeholders, such as through a web site or a specific client. Because participants have different interests, ConQAT can provide a customized view for each stakeholder.

Support trend analysis

Many quality defects are hard to identify in a single system snapshot, but one can see them by tracking changes over time. Moreover, various metrics are hard to interpret on an absolute scale and are better suited for relative measures. To foster trend identification, ConQAT can store and present historical data.

Everything is customizable

Quality requirements are highly project-specific due to differences in target systems, applied tools and processes, involved technologies, and project participants. Moreover, such requirements are not fixed, but rather evolve over the course of a project. Consequently, ConQAT is highly customizable to support project-specific tailoring of both the analysis process and results presentation.

Support diversity

Because diverse elements influence product quality, a quality control tool must apply to many factors and artifacts. ConQAT not only analyzes source code, but also provides measures for other artifacts, such as documentation, models, build scripts, or information stored in change management systems. Also, because stakeholders discuss quality attributes on many different levels, ConQAT facilitates all analysis types at different detail and granularity levels.

Extensibility, autonomous operation and performance

To adjust to yet unsupported artifacts or analyses, ConQAT provides an extension mechanism that lets users add further analysis or assessment modules. To obtain timely results in a cost-effective manner, ConQAT must be easy to integrate into other processes, using fully non-interactive analysis tools. Finally, quality control is particularly relevant for large-scale systems, which are difficult to assess manually. A quality control tool must be capable of analyzing large systems within a reasonable amount of time.

ConQAT Core – A flexible and extensible data-flow language

The core of ConQAT is an interpreter for a data-flow configuration language. The central element of ConQAT’s configuration languages are so-called processors that are interconnected in a pipes-and-filter oriented style. These processors implement diverse functionality and work like functions that accept multiple inputs and produce a single output. This design offers a high degree of reuse and provides a powerful configuration mechanism as processors may be interconnected in numerous ways. To facilitate reuse, configuration fragments can be organized in so-called blocks, which can then be used to build more complex configurations.

Figure 1: ConQAT configurations are constructed from processors and blocks

ConQAT provides an extension mechanism using bundles. A bundle can contribute new processors and blocks to extend the configuration language. The ConQAT distribution itself consists of a large collection of bundles, but using custom bundles, ConQAT's functionality can be extended as needed.

The ConQAT interpreter is a command line application for executing ConQAT configurations. This allows ConQAT to be easily integrated into other processes, such as a continuous build. The output of ConQAT itself is defined by its configuration and as such can be adjusted as needed. Typically, an output of HTML dashboards is used, which can be easily provided to other developers or integrated into other tools, but other output format, including CSV and XML, are possible.

Authoring Tools

To support the creation of complex analysis configurations, ConQAT comes with a set of graphical tools that simplify the creation of blocks and configurations. The Eclipse based IDE provides a library browser for locating available processors and blocks, access to the documentation of these elements, and also refactoring support, to simplify restructuring of configurations. This ConQAT workbench also allows the execution of configurations with a single click, which greatly simplifies testing and debugging of configurations.

Figure 2: The ConQAT workbench is used to create analysis configurations

Analysis Support

The configuration mechanism and the authoring tools are generic and could also be used for other kinds of tools. What makes ConQAT specifically usable for software quality analysis is its huge range of existing building blocks for the creation of such analyses:

• Different input sources: Read from file system, ZIP files, or the version control system
• Different kinds of analysis targets: Analyze source code, models, or word documents
• Different programming languages: Java, C/C++, C#, ABAP, Python, PL/SQL, etc.
• Wide range of analyses: Starting from simple metrics, such as lines of code, to more advanced analyses, such as clone detection and architecture conformance analysis
• Many different ways of aggregating and filtering analysis data
• Flexible output as HTML, CSV, or XML
• Wide range of visualizations, including graphs, treemaps, charts

This set allows a very fine-grained adaptation of an analysis configuration to a specific project or analysis task. This also helps when developing a new kind of analysis, as the developer can focus on the core of the analysis and can reuse input processing, aggregation and visualization from ConQAT.

Dashboards and Inspection Tools

While each of the building blocks of ConQAT might seem very simple at a first glance, this simplicity allows them to be reused and combined in many different ways. By combining these simple parts, one can arrive at rich dashboards that display a variety of analysis results very quickly. Examples of such dashboards are provided in the ConQAT example project that is provided in the ConQAT workbench download.

Figure 3: A simple cloning dashboard created with ConQAT

While these dashboards can provide a good overview of a system's quality, it often can be important to support a more detailed view to better understand the root cause of a quality problem. For this, the dashboards can be extended to link to the source code. Additionally, ConQAT provides graphical inspection tools that integrate with Eclipse and can be used to explore redundancy in source code (code clones) and analyze architecture violations.

Where to go from here?

This article can only provide a first idea of the capabilities of ConQAT. The tutorials provided on ConQAT's web site are a good starting point to better understand ConQAT. There you can also find the reference documentation, which describes many of the existing analysis steps.

---

Click here to view the complete list of tools reviews

This article was originally published in the Spring 2015 issue of Methods & Tools