Reward Of Reinforcement Learning Of Test Optimization For Continuous Integration

Testing in the continuous integration is characterized by constantly changing test sets,limited test time,fast feedback,and so on.It also has strict requirement on time and resource,and there are requirements for large changes in test case sets,limited test time,and fast feedback.Traditional test optimization methods are difficult to apply.Reinforcement learning is an important branch of machine learning.Its essence is to solve sequential decision problems and can be used for test optimization in continuous integration.The test optimization method based on reinforcement learning utilizes the automatic decision-making and adaptive characteristics of the reinforcement learning algorithm,combined with test case prioritization and selection techniques,and is applicable to continuous integration environment.However,in the existing reinforcement learning based methods,the reward function calculation only includes the execution information of the test case in the current integration cycle.We already knew that the historical execution information of the test case can effectively reflect its error detection capability,which is crucial for test optimization.Moreover,Reinforcement Learning itself is learning process based on history.Therefore,this paper is mainly about the reward mechanism in reinforcement learning,focusing on the historical execution information of test cases.We systematically carry out our research from two aspects:reward function design and reward strategy.In the design of reward function,complete historical execution information of the test case is used to replace the current execution information,and we also propose two novel reward function combining the total number of historical failures and historical failure distribution information of the test case.In terms of the reward strategy,we firstly propose two reward strategies,which are overall reward for test cases in current execution sequence and partial reward only for failed test cases.And we propose the optimized reward strategy:?-partial reward which is based on the failure rate of the test case.In order to verify the test optimization effect of the reward mechanism proposed in this paper,experimental research is conducted on four industrial-level programs with four research questions.The results show that:(1)Compared with the existing methods,reinforcement learning methods based on reward function with complete historical information proposed in this paper can help to identify potential failure test cases and greatly improve performance of test optimization in continuous integration;(2)Test case historical failure distribution can help to identify potential failure test cases,which can significantly improve the error detection ability of test sequences;(3)The two reward strategies,i.e.overall reward and partial reward,are influenced by various factors of the system under test,therefore the reward strategy need to be selected according to the actual situation,and adaptive reward based on failure rate of the test case has the best performance in all four datasets;(4)History-based reward functions have longer time consumption,but the test efficiency is not affected,especially for industrial datasets.