An Empirical Research On Input-domain Fault Localization

Thorough testing is required to ensure the quality of a software system.However,in real life,there may be too many parameters to cover all parameters for comprehensive testing.Combinatorial testing is one of the input-domain testing techniques,which can generate smaller test cases to detect faults triggered by the interaction between parameters in a software system.Then input domain fault localization techniques is adopted to find minimal failure-causing schemas(MFSs).These techniques are divided into two types: non-adaptive methods and adaptive methods.The former uses two types of ELA,including high-strength CA and PCA.The latter includes Iter AIFL,FIC,RI,TRT and ICT.This paper improves the shortcomings of previous research that the initial test case suite is uncontrolled and generates it based on real MFSs.Then Siemens program suite,who can adopt parameter values greater than two,is adopted as an experimental object to make a comparison among current existing input-domain fault localization methods.The main contributions of this paper are as follows:(1)ELA assumes that the safe value vector exists and PCA is generated based on the safe value.Then Taking 20 Boolean-specifications extracted from TCAS system and their variants as experimental objects to verify whether the safe value vector is ubiquitous.(2)Taking variants with the safe value vector in the TCAS system as experimental object and evaluate the effectiveness and efficiency of non-adaptive method using high-strength CA or PCA as ELA from recall,precision,f-measure,and the number of test cases required.(3)Taking Boolean-specifications suite and five programs including tontifo,printtokens2.0,printtokens2.2.0,schedule1 and schedule2 of Siemens suite as experimental objects.Then evaluating the performance of adaptive methods such as TRT,FIC,RI,Iter AIFL and SIMP from the number of additional test cases,recall,precision,and f-measure.(4)Taking 20 Boolean-specifications extracted from TCAS system and their variants as experimental objects and evaluate the performance of non-adaptive and adaptive methods from the number of test cases,recall,precision and f-measure.Then compare the fault localization effect of non-adaptive method who adopts CA with strength is t+d as ELA.