A DSL for Defining Feature-Level Quality Constraints and the Aggregation of Evaluation Results in DevOps

Quality requirements typically differ among software features, e.g., due to different usage contexts of the features, different impacts of related quality deficiencies onto overall user satisfaction, or long-term plans of the developing organization. For instance, maintainability requirements might be particularly high for software features which are frequently used or bear strategic value for the developing organization. Also, software features where even the smallest delays are perceived as negative by the user will be subjected to specially tight performance requirements. We defined an operational DSL to define software quality requirements as individual feature-level constraints based on quantitative measures. The DSL provides language elements to define the operationalization of measures from external systems, time series operations, time filters, and the automatic evaluation of these feature-level constraints in DevOps based on comparison operators and threshold values. In addition, quality ratings summarize evaluation results of features on an ordinal grading scheme. Likewise, quality gates use these quality ratings to reflect the fitness of software features or the overall software product using different states. Finally, we show an example based on a widely-adopted secure mobile messaging app that illustrates the interplay of the different DSL elements.