Research On Real-time Path Planning In Highly Uncertain Scene

With the continuous development of artificial intelligence technology,the problem of path planning has become a very mature technology in static,deterministic scenarios.However,in highly uncertain scenes,such as uncertain environmental information and uncertain observation information,the complexity of environmental space,frequent emergencies,and the limited line of sight of agents have brought great challenges to the robustness,generalization and adaptability of path planning algorithms.The problem of path planning for agents with strong spatio-temporal constraints in highly uncertain scenes is a research hotspot in the field of automatic planning technology.How to design a real-time path planning method with strong obstacle avoidance ability and high generalization ability is an important research problem to be solved urgently.In this paper,reinforcement learning is used to study discrete action,continuous action,continuous smooth action and multi-agent path planning.The main work of this paper is as follows:1)Dynamic obstacle avoidance and discrete path planning method based on DQN are studied.Firstly,the least square method is used to predict the short-term future trajectory of dynamic obstacles.Then,DDPG learns and makes decisions in continuous space according to the short-term future trajectory of dynamic obstacles,which reduces the path length and the average turning angle.Finally,the reward value is dynamically adjusted to improve the accuracy of obstacle avoidance and the convergence speed of the algorithm.2)Based on the discrete path planning method proposed in the first study,the problem of continuous path planning method based on DDPG is studied.Firstly,the least square method is used to predict the short-term trajectory of dynamic obstacles.Then DDPG provides agents with the ability to learn and make decisions in continuous space according to the short-term trajectory of dynamic obstacles,which reduces the path length and the average turning angle.Finally,the reward function is set based on the artificial potential field,which improves the convergence speed and accuracy of the hybrid obstacle avoidance algorithm.3)Based on the continuous path planning method proposed in the second study,the smooth path planning method based on DDPG is studied.Firstly,based on the principle of dynamics,MPC(Model Predictive Control)algorithm is used to predict the trajectory of dynamic obstacles.Then,according to the predicted trajectory of dynamic obstacles,DDPG is used to provide agents with the ability to learn and make decisions in continuous space.The simulation results show that this method can accurately realize mixed obstacle avoidance and smooth path planning in dynamic environment.4)Based on the path planning algorithm of single agent proposed in the above three aspects,the space-time path planning algorithm of multi-agent based on MADDPG(Multi-agent Deep Deterministic Policy Gradient)is studied.The algorithm firstly calculates the path of multi-agent in BIT*(Batch Informed Tree*)by multi-threading,then designs a collision detection module in three-dimensional space-time to detect the collision relationship between obstacles and agents in real time,and finally uses MADDPG algorithm to re-plan the path in real time.Experimental results show that the algorithm improves the efficiency and real-time of multi-agent route planning.