Research On Test Case Prioritization Technique Based On History Execution Information In Continuous Integration

As one of the effective practice methods of agile development process model,continuous integration(CI)mainly adopts rapid continuous iteration to deal with frequent requirement changes,and ensures software quality through CI testing.It is infeasible for CI testing to rerun all existing test cases in random order,since it would cause high test resource consumption and a long-feedback cycle.Consequently,improving the effectiveness of regression testing in CI becomes a meaningful research topic,whose objective is to find a technique to implement the quick-feedback of testing result and reduce the testing resource as more as possible,without affecting the overall quality of the software.In this paper,firstly,28 important studies about test case prioritization in CI are selected.Then a systematic review is completed,and it performs the summary in terms of the research objectives,research methods,impact factors and evaluation indicators.Finally,the analysis of the challenges of the existing work is made.The results show that 1)Testing history information is one of the impact factors in 70% of the test case prioritization approaches;2)The multi-objective approach and the application of artificial intelligence(AI)are the main directions of test case optimization in the future.Based on the review,this paper takes the history execution information as the focus of attention and presents the test case prioritization approaches being applied the different phases of CI.The main contributions and innovations of this paper are as follows:1)A test case prioritization technique based on clustering algorithm and defect prediction is proposed.The number of clusters and the choice of clustering features,as well as the inter-cluster prioritization have great influence on the effectiveness of test case prioritization.However,few of the existing test case prioritization approaches based on clustering algorithm discuss them.This paper compares the prioritization results with different numbers of clusters and clustering features under the same clustering algorithm.Then it integrates the fault prediction and clustering methods,and implements inter-cluster prioritization guiding by the results of fault prediction.The experimental results show that the clustering approach can produce the optimal execution order if the optimal number of clusters and clustering features are acquired,otherwise the clustering approach combining fault prediction can similarly improve the fault detection rate,especially in the initial stage of CI testing.2)A test case prioritization technique based on the multi-objective criterions is proposed.Being the various of the software application scenarios,the traditional singleobjective approach is difficult to meet the requirements of the higher fault detection rate and the lower testing resource consumption in CI.This paper presents the multiobjective test case prioritization criterions based on the history execution information,the requirement priority,the execution cost of test case and the switching cost between adjacent test cases.The test case prioritization approach based on history information is treated as baseline.The requirement priority is used to prioritize the test cases which have the same history execution information or the new test cases.The multi-objective problem which is prior executing the test cases with the lower execution cost and switching cost can be converted to the single-objective by weighted summation and greedy algorithm.The effectiveness of test case prioritization approaches are compared based on the dynamic requirements,the fixed requirements and the different length of the history execution information.The experimental results show that the approach combining the history information and requirements has the higher effectiveness if the the history execution information is insufficient.The approach based on the dynamic requirements is more effective,the dynamic adjustment strategy can find the obsolete test case in each cycle which can reduce the test suite size.The higher fault detection rate and the low testing resource consumption can be traded off by the prioritization goals of the execution cost and the switching cost.3)A test case prioritization technique based on the long short-term memory(LSTM)network is proposed.When there are sufficient CI cycles and testing history information,it is difficult to acquire the priority value of test case totally according to the history execution information for traditional deterministic approaches.This paper proposes a novel adaptive approach for automatic prioritization and minimization.The history execution information is specified the time series prediction model that can be trained and predicted using LSTM-based deep learning network.The LSTM prediction value is referred as the probability of detection faults in the next cycle.In order to avoid the decline of performance for the test case augmentation or insufficient historical information(noise data),the hybrid approach is presented combining the deterministic approach and the adaptive approach,the priority values of the noise data are obtained by the deterministic approach.The experimental results show that when the number of CI cycles is up to about 300,the adaptive approach outperforms the other comparison approaches.The hybrid approach can solve the noise interfere which is more valuable for industrial software.