Research On Multi-Agent Coverage Method Based On Deep Reinforcement Learning

The covering problem is one of the important research subjects of Multi-Agent System(MAS).The emergence of deep reinforcement learning algorithm has greatly promoted the development of the multi-agent field.The multi-agent coverage task based on deep reinforcement learning means that the agent can continuously explore the environment and make actions according to the current strategy to get feedback,and then cover the target point or target area in the environment through continuous trial and error.At present,the research on the domain coverage method of single agent is relatively mature,while the research on the domain coverage method of multi-agent is still faced with difficulties and challenges.In this thesis,based on the deep reinforcement learning algorithm,multi-agent target coverage and region coverage are achieved.The main work contents are as follows:(1)Single agent target coverage.Based on the Multi-Agent Particle Environment(MPE),the simulation environment is constructed,and the traditional Artificial Potential Field(APF)method is used to achieve the single agent target coverage.Since the artificial potential field method does not have the ability to learn independently and adapt to the environment,a Deep Deterministic Policy Gradient(DDPG)method is introduced to achieve target coverage,taking the motion information and position information of agents as the state space.The speed information is used as the action space to realize the single agent target coverage based on DDPG method.The simulation results show that the agent can reach the target point quickly by learning the shortest path in the free zone.In the obstacle area,the agent can avoid the obstacle to complete the target coverage task.(2)Multi-agent target coverage.Add artificial virtual potential field values to the state space of Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm.A reward function guided by potential field information is designed to combine the artificial potential field method with MADDPG algorithm.APF-MADDPG algorithm for agent control is proposed to apply to multi-agent target coverage task.On the basis of single-agent target coverage environment,by increasing the number of agents and adjusting the position of target points and obstacles in the environment,the free area and obstacle area are constructed respectively for the situation that the number of agents is 2 and 3,and the two algorithms before and after improvement are used to conduct experiments in the corresponding area.The experimental results show that the convergence effect of the improved algorithm is significantly improved,and the agent can avoid obstacles to complete the target coverage task in the area with obstacles.(3)Multi-agent regional coverage.On the basis of multi-agent target coverage,the multi-agent region coverage method is further studied,and the state space,action space and reward function of multi-agent region coverage are designed.Proximal Policy Optimization(PPO)and action selection optimization were introduced to realize regional coverage of 3 agents.The simulation results show that the improved algorithm has faster convergence speed and better stability than PPO method.At the same time,the regional coverage efficiency is studied by using the method before and after improvement under different number of agents.The experimental results show that when the number of agents is 2,3,4 and 7,the regional coverage ratio of the improved method increases by 4.072%,1.534%,0.278%and 0.030%compared with PPO method.The average number of covering steps decreased by 122.82,269.20,210.64 and 142.96.