Path Planning For Unmanned Vehicles In Unknown Dynamic Environments Based On Deep Reinforcement Learning Algorithm

At present,autonomous driving technology is widely used in the logistics field.Through vehicle networking,automatic navigation systems and obstacle avoidance systems,unmanned vehicles can replace manual labor for logistics transportation of people and goods,further saving logistics transportation costs,improving transportation efficiency and logistics safety.Unmanned vehicle path planning usually uses various sensors to establish high-precision maps in the environment,and performs global planning and local planning on this basis.In outdoor environments where there are fewer obstacles,the maps established by general slam mapping algorithms are prone to large deviations and cannot accurately locate;in indoor environments,there are usually dynamic obstacles such as people and animals and static obstacles such as tables and chairs.It is required that unmanned vehicles have a certain degree of generalization ability in unknown environments,combine perception and decision-making when facing unknown environments,and make quick and correct decisions.To this end,this paper proposes a deep reinforcement learning method to solve the path planning problem in unknown dynamic environments.First,the research status of deep learning,reinforcement learning and vehicle path planning are analyzed respectively.On the basis of mainstream deep learning,reinforcement learning and deep reinforcement learning algorithms,the advantages of various deep reinforcement learning algorithms are combined to improve and propose the APFD3 QNPRE algorithm.In order to enhance its adaptability and generalization ability in the environment,multiple states are used as network inputs,including visual state information,single-line lidar information,and self-state information.Convolutional neural networks are used to process depth visual information;LSTM long-short term memory neural networks are used to process lidar and state information.The output is a speed transmission instruction to guide the movement of unmanned vehicles in unknown environments.In order to ensure that each sample has the opportunity to be sampled and reduce the number of training times in unnecessary states,a replay mechanism based on sample experience reward value and resampling is proposed in the experience replay part,which further improves the convergence speed of the algorithm.In response to the problem that the algorithm cannot adapt well to complex dynamic environments,an action output mechanism based on APF is proposed.The output of APF is used as prior information and another state input of the network.In the output part,the improved DQN algorithm is combined for the final action output of unmanned vehicles.Secondly,based on the ROS operating system and gazebo simulation platform,a simulation environment is built.Based on the simulation environment,design corresponding reward functions and confirm specific values of algorithm-related parameters.Comparative experiments were carried out with mainstream deep reinforcement learning algorithms and traditional map-based path planning algorithms.Finally,in order to further verify its practical application effect,a real vehicle environment was built to verify the path planning effect of unmanned vehicles.Through simulation and real vehicle verification,the APF-D3 QNPRE algorithm has good generalization ability in simulation environment and has significant advantages over other algorithms in terms of convergence speed,loss value,path planning time and length;in real vehicle environment,it can stably perform path planning and obstacle avoidance,providing reliable experimental and application support for actual unmanned vehicle path planning.This paper studies the path planning problem of unmanned vehicles in unknown dynamic environments,uses multiple sensors as state inputs,combines improved deep reinforcement learning algorithms with artificial potential field methods,and ensures the generalization,obstacle avoidance ability and planning effect of unmanned vehicles in unknown environments.It has certain research significance and application value for the promotion and application of unmanned vehicles in the field of logistics distribution.