Research On Reinforcement Learning Method For Game Manipulation Behavior Imitation

In recent years,many AI developed in the game environment shine.Games provide an environment for AI to test the model,and the success of AI in turn drives the development of games.They can form a complementary relationship.The imitation learning method learns the given expert trajectory,so its portability is very high.After the progress of imitation learning in the game,it can be easily applied to other fields.This thesis studies the application and improvement methods of imitation learning in the image-based mobile game environment.The main work includes the following aspects:(1)For the problem of low sample utilization rate of GAIL method,the SAC algorithm based on Q-Learning is proposed to improve it,and the application method of SAC in GAIL is deduced.The strategy iteration of the original GAIL algorithm estimates the value function,so it can not use the empirical playback technique of off trajectory strategy to improve the sample utilization.At the same time,the energy model strategy based on maximum entropy makes the strategy promotion more stable and robust.(2)An expert trajectory guidance strategy search method is proposed.In imitation learning,the expert trajectory is given directly.Because GAIL algorithm is improved to be an off trajectory strategy method in this thesis,the strategy can be trained through experience playback,so the expert trajectory is added to the experience playback pool.It can not only improve the exploration efficiency of agents,but also improve the stability of the algorithm.(3)An image feature extraction model for mobile game environment is proposed.Based on vision transformer,convolution network features and variational self encoder features are embedded.They help the model extract local features and implicit features respectively,which complement the global feature extraction ability of Vi T.Considering the particularity of the mobile game environment,the original image block is replaced with as little influence as possible,so that the model can be obtained by fine-tuning migration.Finally,taking Cool running every day game as the experimental environment,the effectiveness of the method proposed in this thesis is verified.