Study On Effectiveness Of Metamorphic Relation

In traditional software testing,testers often need to build test oracles to verify whether outputs of the program are right or not.Nevertheless,test oracles are difficult to obtain or even can't obtain in most cases,which is the oracle problem in software testing.However,metamorphic testing effectively alleviates the oracle problem by comparing many execution results of the program whether satisfy some specific relationships or not.The most important part of metamorphic testing is building metamorphic relations,which rely on professional knowledge in the domain which the program is to be measured.On one hand,the paper proposes a metamorphic relation applied on compiler testing named ECPMR and a method by verifying whether program slices satisfy the ECPMR to test the compiler on the basis of ECPMR.The method firstly selects some correct programs which satisfy specific metamorphic relation as inputs of the compiler under testing.Then,use those selected programs to generate program slices.Finally,check outputs of program slices whether satisfy ECPMR or not to find bugs in the compiler.Due to building metamorphic relation needs relying on professional knowledge of the specific domain and the number of this method is limited,combination of metamorphic relation comes.However,how to select a better quality of metamorphic relation in order to achieve a higher error detection rate has become an urgent problem.So the other hand of the paper is a study on effectiveness of metamorphic relation.Based on the preliminary analysis of the influencing factors of effectiveness of metamorphic relations,we classify metamorphic relations by the number of variables and the number of function points.According to the number of variables,we define single variable metamorphic relation,two variable metamorphic relation and multivariable metamorphic relation.We divide the metamorphic relation according tothe number of function points in the program covered by the metamorphic relation,and define the single functional metamorphic relation and the multifunctional metamorphic relation.Then though several groups of experimental research,we concluded that the error detection rate of the most metamorphic relations increases with the number of variables increasing,that is to say the multivariable metamorphic relation is more effective than single variable metamorphic relation and two variable metamorphic relation.But influenced by the false negative problem,there are also some multivariable metamorphic relations worse than two variable metamorphic relation.Because the multifunctional metamorphic relation covers more function points and the program is performed in more extensive coverage,the multifunctional metamorphic relation has a higher error detection rate than the single function metamorphic relation.In addition,in order to alleviate the false negative problem,the more metamorphic relation is proposed,which is to add more constraints to the general metamorphic relation,so as to reduce the occurrence of the false negative problem.Through the experimental study,we further verified the more metamorphic relation has a higher error detection rate than the general metamorphic relation.