Research On Reward Function Of Reinforcement Learning In Continuous Integration Testing

Continuous Integration(CI)requires frequent completion of software integration build and test to respond to business delivery in time,and the rapid feedback mechanism of CI brings new challenges to testing.Test Case Prioritization(TCP)based on reinforcement learning has been practically applied in CI testing,where reward is an important component of reinforcement learning and a good reward function plays a crucial role in the effectiveness of reinforcement learning.In the study of TCP reinforcement learning reward for CI testing,the reward function based on the test case historical execution information has been widely studied,focusing on two factors:the historical failure quantities and the failure distribution of test case.The higher the historical failure quantities,the more proximate the failure results are,indicating that the test cases have greater defect detection capability.Existing studies of reward functions usually considered the effect of a single factor,either with the failure quantities as the benchmark,or focusing on the failure distribution but ignoring the effect of the failure quantities on the defect detection capability.This paper considers both factors and studies the reward function further based on the information of the failure distribution combined with the consideration of the influence of the failure quantities on test optimization.This paper focused on reinforcement learning reward function in CI testing,considering the failure quantities and failure distribution information of test case together,and proposed the exponential weighted fusion method and quadratic weighted fusion method based on failure position for different weighted methods.Based on the above two methods,Average Position Exponential Weight Reward(APEW)and Average Position Quadratic Weight Reward(APQW)were designed respectively by combining the historical execution failure information of test cases.In addition,Average Position Exponential Weight with Pass Information Reward(APEWP)and Average Position Quadratic Weight with Pass Information Reward(APQWP)combining the historical pass information and failure information of test cases were designed respectively by taking test cases’ pass information into consideration.In order to evaluate the test optimization effect of the reward function proposed in this paper,an experimental study was conducted on ten industrial datasets.The results showed that:(1)APEW and APQW reward functions based on test case historical failure information improved 2.22%and 1.01%,respectively,on NAPFD compared to APHF reward.They improved 3.58%and 3.48%on RECALL,respectively,effectively improving the defect detection capability and coverage of test sequences.(2)APEWP and APQWP reward functions combined with test case historical pass information improved 3.10%and 8.41%on NAPFD and 4.14%and 8.72%on RECALL,respectively,compared to APHF reward,further improving defect detection capability and coverage.(3)The time overheads of the four newly proposed reward functions were compared,which shortened the testing time and improved the testing efficiency compared to APHF reward.