Research On Multi-Agent Collaborative Algorithm Based On Reinforcement Learning

With the development of scientific technology,artificial intelligence(AI)is becoming one of the hot research areas,and it is the basis of modern AI research to construct intelligent agents that can make better policies under complicated environment.However,due to the partial observability,instability of environment,and the mutual interaction and restriction among multiple agents,the multi-agent problem is still a very challenging task.One of the better methods for tackling the multi-agent problem is multi-agent collaboration based on reinforcement learning.Communication among multiple agents is one of the most common ways to solve collaboration problems.However,most communication protocols among agents use manual methods to deliver static communication content,fail to capture dynamic interactions among agents,and result in instable environment.To solve this problem,in this thesis,a new Attentional Communication Model(ACM)is proposed to dynamically achieve the cooperation among multiple agents with training agents as fast as possible by adaptively constructing communication routes and messages.The main work and contributions of this thesis include:(1)To propose a new Collaboration Awareness Network(CAN).The novel CAN can dynamically calculate the relationships among agents to ascertain the routing and distill the state information into the message.It not only saves the communication resources but also makes better use of the action strategy information so that the agents can get smart cooperation strategies and improve the stability of training.Most importantly,our CAN dynamically builds communication protocols to adapt to changing environments and strategies.(2)To construct a new ACM.A new ACM is built by integrating our CAN with the policy network built by reinforcement learning algorithms.The two networks iteratively update to obtain the agents with collaboration ability.In order to shorten training process,the attention mechanism is introduced into ACM to select valid messages,and the meta-learning mechanism is adopted to train agents how to learn.Our ACM presents outstanding ability in collaboration with the environment after sufficient training.(3)A series of experiments are carried out for comparing our ACM with the representative algorithms in both discrete and continuous environments,and the dynamically distilled communication messages are visualized for analysis pursuit game and the continuous environment multi-walker game.The test results in the discrete pursuit game and the continuous environment multi-walker game show that our ACM is better than the benchmark algorithms.