| In any non-trivial software system, reliability cannot be determined exactly. Instead, we must apply statistical methods to create an estimate based on a sample of test cases. This dissertation introduces such an estimate which involves partitioning the population of test cases and selecting samples from within these partitions. These estimates are, however, are bound by constraints on the time and money that an organization is willing to expend to attain them. In this work, two specific constraints are considered, and for each constraint, iterative approaches to test selection are discussed with the goal being to create the most accurate reliability prediction possible given the relevant constraint.; In contrast to fixed sampling schemes, where the proportion of test cases taken from each partition is determined before reliability testing begins, we make allocation decisions dynamically throughout the testing process. Using a Bayesian approach, we can take advantage of information from previous functional testing and insights from developers. We then refine these estimates in an iterative manner as we sample.; Here we compare the results of our sequential sampling schemes and demonstrate their superiority over the optimal fixed sampling scheme in terms of the Bayes risk both theoretically and through Monte Carlo simulations. |
