| The aim of testing is to find errors as early and as cheaply as possible. Specifically, system testing should uncover more errors before release and prevent the release of products that would result in discovering many post-release errors, which cost more to fix than errors found earlier in the life cycle. The effectiveness and efficiency of system testing depends on many factors, not only the expertise of the testers and the techniques they employ. This dissertation develops an integrated method using various techniques that will improve testing effectiveness and efficiency. Some techniques already exist, but are applied in a new way. Others are new.; The integrated method traces post-release problems to one or more factors that influence system testing efficacy. Development defect data help identify which parts of the software should be tested more intensely and earlier. Based on assessment results, testers can develop guidelines to make system test more effective.; During system testing, testers may need to determine quantitatively whether to continue testing or to stop, recommending release. Early stopping could decrease the cost of testing, but may miss problems that would have been detected had testing continued. This dissertation develops empirical techniques to determine stopping points during testing. It proposes a new selection method for software reliability growth model(s) that can be used to make release decisions.; Quality assessment of multiple releases of the same product forms the basis of longitudinal decisions. Techniques using data from prior releases help to identify parts of the system that are consistently problematic. This information aids in developing additional testing guidelines for future releases of the product.; A case study compares and evaluates these techniques using project data from a large software product from industry. It demonstrates that the integrated method can improve the effectiveness and efficiency of system test. It identified problematic software components using data from prior releases and development. Testing these components earlier reveal more defects earlier. Development should, therefore, have more time to fix these defects before release. The method was also able to estimate remaining defect content to make release decisions. |
