Empirical Evaluation of Software Quality Attributes

Why has there been so little progress in the quantitative evaluation of those quality attributes concerned with software design? This paper seeks to provide an answer by arguing that progress is dependent on the adoption of a programme of cumulative empirical research. Such a programme consists of original empirical research, independent replication to investigate and verify original results, and, critically, recipe-improvement replication in an attempt to produce more generalized results. Furthermore, a programme of cumulative empirical research is dependent upon quality attributes being communally de ned and soundly related to appropriate measures. Empirical research is necessary to demonstrate that quality attributes, and their quanti cation, are well-founded. Replication is necessary to investigate and con rm original results. Thereafter, recipe-improvement addresses the signi cant problems of subject, environment and domain variability associated with the empirical study of software quality attributes. Recipe-improvement research is essential if results are to be progressed from the speci c to the general. To support this argument this paper critically reviews the empirical research investigating the quality attribute of module coupling. Module coupling has been advocated for at least two decades. As an attribute of software quality, it is theoretically convincing, generally accepted, and widely taught. Furthermore, it is one of the few quality attributes that have been subject to a number of well publicised empirical studies. The empirical evidence accumulating from these studies, however, is shown to be insubstantial. The key empirical studies to date have di ered in their de nition of module coupling, the e ects sought, the operational measures used, and the results found. It is concluded that this series of single shot studies must be replaced by a programme of cumulative research. Department of Computer Science, University of Strathclyde, Livingstone Tower, Richmond Street, Glasgow G1 1XH, U.K., +44 (0)141 552 4400, [email protected] RR-95-173 [EFoCS-9-95] University of Strathclyde