CISQ Supplements ISO/IEC 25000 Series with Automated Quality Characteristic Measures



The ISO/IEC 25000 series of standards, also known as SQuaRE (System and Software Quality Requirements and Evaluation), contains a framework to evaluate software product quality. ISO/IEC 25010 defines a set of eight software quality characteristics, or system “-ilities,” i.e. security, reliability, and maintainability. ISO/IEC 25023 describes how to apply the quality characteristics to measure product quality. However, the measures defined in 25023 largely measure quality at the behavioral level rather than at the level of specific quality problems in the source code. To supplement the level of measurement in 25023, CISQ has defined source code level measures of four quality characteristics—Reliability, Performance Efficiency, Security, and Maintainability.


Automated software quality measurement is necessary because manual review is infeasible for large multi-layer, multi-language, multi-platform systems. These standard measures can be used for quality assurance, benchmarking, system risk analysis, contract SLAs and software acceptance criteria.


ISO/IEC 25000 is Starting Point for CISQ Work


The ISO/IEC 25010 standard provides consistent terminology for “specifying, measuring and evaluating system and software product quality” (ISO). This figure illustrates the eight software quality characteristics defined by ISO. The four characteristics highlighted in blue were selected and prioritized by the CISQ executive board for automated measurement.



ISO/IEC 25010 Quality Characteristics



CISQ Automates Measurement of Software Quality Characteristics


The four CISQ Automated Quality Characteristic Measures are compliant with the ISO 25010 definitions of quality characteristics. Their sub-characteristics were used to determine the scope to be covered by each measure. The CISQ measures are defined as the sum of critical weaknesses in software that cause the undesirable behaviors underlying many of the measures defined in ISO 25023. These behaviors include such measures as downtime, performance degradation, and amount of data stolen. The weaknesses incorporated into the CISQ measures can be detected from analyzing source code. Collectively, the CISQ measures cover eighty-six critical code quality rules at the code unit and system level.


The Automated Quality Characteristic Measures became OMG® approved standards in 2015. The standards are available for download on the CISQ website or OMG website under Modernization Specifications.



Quality Characteristic Measures to Automate in the Future


CISQ plans to develop automated measures for the ISO quality characteristics of Compatibility and Portability due to the rise of mobile technology. CISQ will also extend its existing quality metrics to cover embedded and real-time systems. The combined measurement of both business and embedded software is critical in IoT (Internet of Things).