Research On Multi-agent Cooperation Method Based On Deep Reinforcement Learning

In the real world,many tasks such as traffic control and military command decisionmaking can be regarded as multi-agent problems.Reinforcement learning is flexible and does not require manual intervention.It is often used to solve problems related to sequential decision-making and has been widely used.Reinforcement learning methods are generally directly applied to single-agent scenarios,and introducing them into multi-agent scenarios will cause many problems such as non-stationary environment.Establishing a communication mechanism between multiple agents can alleviate some of the problems,and at the same time can strengthen the collaboration between agents,thereby generating greater group benefits.This dissertation studies the multi-agent reinforcement learning method with communication mechanism to improve the performance of collaboration between multi-agents.Aiming at the problem of non-stationary environment in multi-agent reinforcement learning,this dissertation uses a multi-agent network model framework with communication mechanism to model multiple agents.The framework allows the fusion of shared communication information provided by each agent in a multi-agent scenario,and the model parameters of the framework can be updated during the interaction between the agent and the environment,so as to learn specific communication mechanisms and improve the agent.Collaborative decision-making ability.In the communication information processing module of this framework,in view of the problems that traditional communication information fusion methods cannot adapt to the changing scenes of agents and the fusion mode is too simple,this dissertation proposes a communication information fusion method based on the attention mechanism.This method allows agents to learn the proportions of communication information from different sources,and to fuse the communication information according to the proportions,so as to better describe the relationship between agents.In addition,this dissertation proposes a hierarchical attention mechanism for the problem that the agent's uninterrupted communication in the communication information fusion method based on the traditional attention mechanism may reduce its own decision-making level.In the communication information fusion method based on the hierarchical attention mechanism,each agent can decide whether to accept communication according to its own local coding information and the information fusion result based on the traditional attention mechanism,thereby learning more flexible and efficient communication modes.This dissertation proposes an overall multi-agent network model and learning method based on the above two communication information fusion methods,combining the deterministic strategy gradient algorithm with the communication information fusion method based on the attention mechanism.In addition,this article optimizes the training method for problems such as the large variance of the Q value estimation during the training process of the deterministic strategy gradient algorithm.At the same time,the multi-agent reinforcement learning algorithm oriented to curriculum learning is obtained with a small amount of modification,so that the intelligent body can learn transferable strategies in the course learning tasks where the agent changes.Finally,this dissertation verifies the effectiveness of the algorithm in multi-agent collaboration scenarios through traffic congestion control experiments,coverage control experiments and course learning experiments.