A Research On Combinatorial Testing Failure Rule

Software testing run through the entire software development cycle, which takes more than40% of its workload. It is an important method to guarantee software quality. Software testing is a process that aims to find the error of software and execute software under test(SUT). It chooses different test case generation strategies carefully in order to generate test cases, then uses these test cases to find errors. In software testing process, the scale and the detection error ability of test suite determine the software testing effect directly. Therefore, how to choose a small scale and high error detection ability test suite is the core and key subject to software testing.The key observation in combinatorial testing is that most software failures are caused by interactions of only a few input parameters. A τ-way combinatorial test set is built to cover all theτ-way interactions, where t is typically a small integer. If test parameters and values are properly modeled, a τ-way test set is able to expose all failures that involve no more than t parameters. The combinatorial testing has the ability to ensure the rate of detecting failures triggered by the interaction of certain factors via fewer test cases. Combinatorial testing has been widely utilized in testing software, e.g. Siemens Suite.This paper aims to investigate the reason why combinatorial testing works in Siemens Suite.Experiments are designed to get the MFS(minimal failure-causing schema) for Siemens Suite,which has been used as a benchmark to evaluate the effectiveness of many testing techniques. The low bound of fault-detecting probability of τ-way combinatorial test suite for each program is calculated by analyzing the strengths and the number of MFS for each faulty version.Computational results could explain the effectiveness of combinatorial testing in Siemens Suite.