| The high expense of software testing processes exposes the fact that the existing testing techniques are neither efficient nor effective. Software engineering researchers have enough motivation to design more rigorous and effective yet less expensive software testing techniques. Experimentation is the primary tool to assess the effectiveness of a set of techniques. To conduct a scientific and replicable experiment, we need a set of subject programs and their artifacts that are well-prepared. Setting up a repeatable and automated experiment with significant volume of data is a challenge.;First, we investigate the behavior of mutants and faults in various circumstances. Research shows that for random-based test suites mutants are good representatives for real faults. We extend previous research by studying their similarities for coverage-based test suites.;Second, the thesis investigates the feasibility of using mutation in assessing the cost-effectiveness of testing techniques. Addressing the trade-off in cost-effectiveness of different criteria assists a test practitioner in choosing the most appropriate criterion.;Third, we tackle the infeasibility of mutation. Mutation is an expensive technique mainly caused by generating enormous number of mutants. The sufficient set problem is the problem of finding a smaller subset of mutants while maintaining the properties of the entire set of mutants. We determine the sufficient set for both random and coverage-based test suites.;Mutation is a fault-based technique by which we generate a set of slightly altered and possibly faulty versions (mutants). Mutation operators are a set of mathematically well-defined transformations simulating simple faults. We tune the quality of mutants by designing various types of mutation operators. Applying mutation operators, we are able to generate a considerably large number of mutants. These properties of mutation make it a suitable tool for conducting repeatable experiments. The idea, albeit interesting, opens up several challenging research questions.;The main theme of this thesis is empirical study and statistical analysis. The results show that mutants can be a good representative of real faults. We adapt several statistical techniques such as variable reduction and subset model selection for determining a sufficient set. The thesis introduces Cost-based Least Angle Regression (CbLARS) by which a set of best fitted regression models are constructed.;Keywords. Testing and Debugging, Mutation Analysis, Mutation Testing, Testing Effectiveness, Experimental Design, Testing Strategies, Test Coverage of Code, Statistical Analysis... |
