Obstacle Avoidance Control For Multi-agent Systems Based On Deep Reinforcement Learning

Given the traditional multi-agent system obstacle avoidance algorithm in efficiency of obstacle avoidance and the obstacle avoidance path time compared with intelligent obstacle avoidance algorithm slightly disadvantage,a growing number of multi-agent system obstacle avoidance algorithm for researchers to focus on intelligent obstacle avoidance algorithm,the research object is expanded from mature single agent obstacle avoidance to the multi-agent system formation of obstacle avoidance.At present,intelligent obstacle avoidance algorithm based on deep reinforcement learning is a research hotspot of multiagent system.However,in the formation obstacle avoidance process of multi-agent system,the deep reinforcement learning algorithm will have a series of problems,such as too many training steps or the effect of obstacle avoidance is not obvious.In order to solve the problems mentioned in the application of the intelligent obstacle avoidance algorithm in multi-agent system,the following studies are made in this paper:First of all,on a single agent of obstacle avoidance algorithm,based on Deep Deterministic Policy Gradient algorithm on the basis of the traditional reward function was improved,according to the real environment of obstacle avoidance of agent in static obstacles for simulation environment,In the complex obstacle environment,the single agent carries out the simulation comparison test of traditional DDPG algorithm and the improved DDPG algorithm.Through the comparison of the simulation result graph and the loss function under different training times,it is concluded that the path obtained by the improved DDPG algorithm is smoother and the obstacle avoidance effect is better.Secondly,for the obstacle avoidance algorithm of multi-agent system formation,in view of the formation instability in the path planning process of multi-agent system formation,the multi-agent formation mode is improved,and the formation is carried out by adopting the Angle distance measurement.Aiming at the problem that multi-agent system is slow in acquiring path time,the path planning method formed by the traditional obstacle avoidance process from starting point to end point is changed,the concept of starting point and end point is weakened,and the way of adding midpoint in path is added,so that the agent can move from starting point and end point to the midpoint simultaneously.Forming two paths from start point to middle point and endpoint to middle point;Aiming at the problem that the multi-agent system is close to the obstacle in the process of obstacle avoidance,the reward function is designed as the positive reward when the agent of the same sign collided with the obstacle,and the negative reward when the agent of different sign collided with the obstacle.Through the verification of the improved DDPG algorithm in the static obstacle and dynamic obstacle simulation environment,the simulation result graph and the comparison of the reward value under different training times show that the improved DDPG formation algorithm saves the path acquisition time compared with the traditional DDPG algorithm,and the formation obstacle avoidance effect is more obvious.Finally,in the framework of machine learning library Tensor Flow,the obstacle avoidance process of a single agent and the obstacle avoidance process of multi-agent formation are simulated.The results show that in the simulation of single agent obstacle avoidance,the improved DDPG algorithm is better than the traditional artificial potential field method and the original DDPG algorithm in the effect of obstacle avoidance,and the path obtained is more smooth.In the simulation of multi-agent system formation obstacle avoidance,the improved algorithm keeps stable formation in the process of path planning,and the effect of obstacle avoidance is obvious.