| Software testing,which has already been widely studied and applied in both academia and industry,is an effective investigation conducted to provide infor-mation of the quality of the software or service under testing.Among many software testing techniques,Combinatorial Testing(CT)is an important black-box method.It usually generates test suite according to a mathematical object,i.e.,covering array,and has been proven effective in revealing the failures caused by the interaction of factors that affect the behavior of a system.Many studies in CT focus on the algorithms of generating test cases,or the applications of CT into practice.However,few studies focus on the analysis of the outcomes of the test cases after generation and execution,which leads to the lack of understandings of the testing.One specific questions is,even though we detect some failing test cases,we do not know which one or some interactions of factors are the causes of the fault.As a consequence,CT does not give enough support to real software testing scenario.This is because fault diagnosis and fault fixing works are usually labor intensive in practice.The lack of further analysis of the outcomes of the test cases would let the testers and developers check many unnecessary and invalid interactions,which surely adds additional burdens.According to these facts,this thesis focuses on the foundation of the analysis of testing outcomes,i.e.,the minimal failure-causing schemas(MFS).MFS is the minimal interaction of factors that can cause a failure.All the other interactions of factors which can trigger failures can be obtained based on MFS.Hence,the identification of MFS can greatly improve the effectiveness and correctness of fault diagnosis,which benefits the bug fixing works and offers support,for the control of the quality of the software under test.For all of these,the study of MFS is of great importance.In this thesis,we introduce the definition,formal properties,and the applications of MFS.After that,we summary some important problems and restrictions in this field,and offer the corresponding solutions.The contributions of this thesis include:1)We refine the formal definition of the Minimal Failure-causing Schema(MFS),and give some important properties of it,which include the existence of the MFS,the relationship between MFS and faulty or non-faulty schemas,the relationship between MFS and test cases.These basic properties offer the theoretical foundations for the following studies.2)We propose a novel MFS identification framework.This framework con-structs a candidate tuple relationship tree(TRT)to describe the relationships among all the candidate factor interactions.TRT facilitates our localizing progress by reducing additional test cases generated and providing a clear view of all pos-sible candidate interactions so that any MFS,such as overlapped MFS,would not be missed.And based on TRT,we propose five search methods to real-ize fault-localizing progress,i.e,random selection,depth-first,width-first,greedy and path-based search.3)We systematically deal with the masking problem between multiple MFS.The situation of masking effects between different faults,i.e.,the scenario that one fault may be prevented to be observed because of the occurrence of another fault,challenges the traditional MFS identification.This thesis theoretically studies the impact of this effect on the identification of MFS.One conclusion is that,even though we exhaustively execute all the possible test cases,we cannot completely remove this negative effect.Moreover,we offer an elemental method,i.e.,test case replacement method,to alleviate this effect.4)We propose and solve the problem that how to make both the MFS iden-tification and test case generation processes more effective and efficient.To this aim,we propose a novel framework,i.e.,interleaving CT,to interact with each other more closely.This framework reduces the generation of redundant schemas and test cases,and improves the quality of MFS identification.We remodel the test case generation and MFS identification modules to make them better adapt to this new framework.Specifically,for the generation part of our framework,we augment it by forbidding the appearance of test,cases which contain the identi-fied MFS.such that it will reduce the generation of unnecessary test cases that does not contribute for additional failure detection.For the MFS identification module,we augment it to achieve higher coverage.As a result,both generation and identification stages will be done more effectively and efficiently.5)With respect to two important restrictions of the MFS identification,i.e.,safe value assumption and determined failure assumption,we give correspond-ing analysis and discussion.In the traditional MFS identification approaches,researchers need to import two assumptions,i.e.,the existence of safe values and all the failures must be determined,to make the MFS identification process more effective and efficient.In this thesis,we study the negative influences when these two assumptions do not hold.Additionally,for these two problems,we give two corresponding solutions,i.e.,adding feed-back checking process and increasing the number of test case execution times,respe.ctively,to alleviate the negative effects. |
