A MEASURE OF TEST CASE COMPLETENESS (SOFTWARE, ENGINEERING)

When testing a computer program, it is not usually clear when to stop testing and, at the same time, have some level of assurance that the program is correct. This decision, as well as the selection of test cases, is often done in an ad hoc manner.;It is also shown that the cause-effect graphing and equivalent normal form approaches to test case generation produce the same test cases. The equivalent normal form method is more algorithmic and easier to implement.;This dissertation addresses a technique that can be used to gain assurance that the quality of test cases are improving. First, a metric is developed to measure the effectiveness of a set of test cases developed using a particular testing approach, such as statement coverage, branch coverage, multiple condition coverage, path testing, cause-effect graphing and mutation analysis. A single measurement approach for test cases is developed, regardless of the test approach used to generate the test cases. This metric is used to evaluate both structural and functional methods for generating test cases. Next, a composite metric is constructed based on metrics developed for the approaches that were evaluated. This composite metric is shown, for the examples studied, to increase as the number of errors discovered increases. That is, by adding more test cases in a manner that increases the composite metric, one is likely to find more program errors than by adding test cases in a random manner. In addition, by applying regression analysis, some of the components of the composite metric are shown to be a predictor of the reliability of the programs in the sample studied.