Researches On Generative Adversarial Imitation Learning For Multi-Agent Games

Controlling multiple agents to complete cooperative or competitive tasks is a common problem in the fields of autonomous driving,robot control,game and television.As software that can organize mltimedia,video games are often used in multi-agent environment simulation and research in various fields.The study of multi-agent problems in games is of universal significance to related industries.With the development of computer technology,deep reinforcement learning has achieved remarkable achievements in various human-machine game.But in the game development,the game agent should not only meet the engineering requirements but also serve the users.The Agent often requires more features such as high efficiency,controllable difficulty,and anthropomorphic operation.Compared with reinforcement learning,imitation learning is a supervised learning algorithm,which has the characteristics of high training efficiency,controllable difficulty and anthropomorphic actions.How to use imitation learning to simplify development and improve user experience is great significance to industry.This thesis takes the application of imitation learning in multi-agent games as the research theme.Aiming at the performance and efficiency of imitation learning in the application of multi-agent games,this proposes a centralized training decentralized decision-making multi-agent generation anti-imitation learning algorithm that introduces a multi-head self-attention and verifies it.Finally,this thesis verifies the performance and efficiency of the algorithm controlling the agent in the actual game by building an action game and applying the improved algorithm,and further compares and studies the characteristics of the improved algorithm and the behavior tree algorithm applied in the development of game agents.The work of this thesis can be detailed as follows:(1)For the multi-agent games application requirements,this thesis compares and analyzes the algorithms of multi-agent imitation learning,and proposes a generative confrontation imitation learning algorithm for multi-agent learning.The algorithm is based on generative confrontational imitation learning,and adopts the structure of centralized training and decentralized decision-making.In this way,the model can obtain balanced training and decision-making efficiency.(2)This thesis adds a multi-head self-attention to improve the representation learning ability of the model discriminator on the basis of the improved structure of the generative confrontation imitation learning algorithm.The improved model in this thesis can better utilize the state information of each agent in the multi-agent,thereby improving the performance and stability of the algorithm.(3)This thesis designs a multi-agent game that is closer to modern games.Compared with the experimental environment,it has more complex elements,such as actions,skill props,and randomness.In the game,the agent based on the improved algorithm and the behavior tree algorithm is designed,,and compares and studies the application performance and characteristics of the improved algorithm compared with the traditional behavior tree algorithm in the game development process.