Reflection on Software Quality

McCall’s (1977) innovation was the creation of a relationship between quality and metrics. McCall covers all aspects of software development in a way that is, in my opinion, still actual for the best part. McCall shows its age essentially in how it covers the software lifecycle because it embeds the waterfall process in its structure. McCall also focuses on technological aspects such as portability and standards that are still actual, but largely resolved by standardized frameworks and technologies that are inherently platform-independent.

Modern models created after 2001 diverge from older models such as McCall or ISO 9126 because they are tailored for specific applications (Miguel et al. 2014). An example of these models is SQO-OSS tailored to evaluate OSS.

SQO-OSS focuses on the source code and includes the community in the overall evaluation. It is purely metric-oriented and measurements can, in theory, be automated. Such metrics are translated into one of several qualitative categories. The code is measured with standardized metrics such as cyclomatic complexity that many tools should be able to support. The community instead is measured on metrics such as the number of messages in the support channel (Samoladas et al, 2008).

Being largely automatable SQO-OSS differs from McCall requiring instead a case-by-case evaluation. Thanks to automation, SQO-OSS could provide an always up-to-date score, while McCall cannot be updated without considerable effort. However, the dependency of SQO-OSS on specific tools to measure the code may be problematic for the adoption of this model (Madaehoh et Senivongse, 2022), and also the measurement of the community’s quality is subject to the availability of the data (Izquierdo-Cortazar et al, 2010). McCall instead is universal since it relies only on general principles. Another difference is in terms of precision: SQO-OSS uses multiple qualitative categories from excellent to poor while McCall’s measurement is inaccurate since it relies on many binary measurements (Miguel, 2014).

I doubt that any methodology today can be widely adopted without automatic tools able to effortlessly provide a synthetic evaluation always up-to-date. McCall remains a milestone in code quality that should be incorporated into software development guidelines but is inapplicable nowadays.

References

Izquierdo-Cortazar, D., Gonzalez-Barahona, J. M., Duenas, S., & Robles, G. (2010, September) Towards automated quality models for software development communities: The QualOSS and FLOSSMetrics case. In 2010 seventh international conference on the quality of information and communications technology (pp. 364-369). IEEE. Available from https://www.researchgate.net/profile/Gregorio-Robles/publication/224202217_Towards_Automated_Quality_Models_for_Software_Development_Communities_The_QualOSS_and_FLOSSMetrics_Case/links/00463526ff03a63980000000/Towards-Automated-Quality-Models-for-Software-Development-Communities-The-QualOSS-and-FLOSSMetrics-Case.pdf

Madaehoh, A., & Senivongse, T. (2022, November) OSS-AQM: An Open-Source Software Quality Model for Automated Quality Measurement. In 2022 International Conference on Data and Software Engineering (ICoDSE) (pp. 126-131). IEEE. DOI: 10.1109/ICoDSE56892.2022.9972135

McCall, J. A., Richards, P. K., & Walters, G. F. (1977) Factors in Software Quality, Concept and Definitions of Software Quality. General Electric Company. Available from https://apps.dtic.mil/sti/pdfs/ADA049014.pdf

Miguel, J. P., Mauricio, D., & Rodríguez, G. (2014). A review of software quality models for the evaluation of software products. arXiv preprint arXiv:1412.2977. Available from https://arxiv.org/pdf/1412.2977.pdf

Samoladas, I., Gousios, G., Spinellis, D., & Stamelos, I. (2008). The SQO-OSS quality model: measurement based open source software evaluation. In Open Source Development, Communities and Quality: IFIP 20 th World Computer Congress, Working Group 2.3 on Open Source Software, September 7-10, 2008, Milano, Italy 4 (pp. 237-248). Springer US. Available from https://gousios.org/pub/sqo-oss-quality-model-measurement-based-open-source-software-evaluation.pdf