Research On Obstacle Avoidance And Tracking Algorithm Of UAV Based On Deep Reinforcement Learning

In recent years,the application field of unmanned aerial vehicle(UAV)is expanding with the development of science and technology.Among many task types of UAV,the realization of autonomous obstacle avoidance and target tracking has always been one of the research priorities.At present,there are mainly non-machine learning algorithms for the obstacle avoidance and tracking problem of UAV.Some of these algorithms need to manually adjust a large number of parameters,some need to build a three-dimensional model of the scene.Besides,there are algorithms based on supervised learning.These algorithms need to collect a large number of labels for network training.However,these algorithms based on deep reinforcement learning overcome the defects of the above algorithms.They can model the obstacle avoidance and tracking problem of UAV into a decision-making process,and realize an end-toend output.Based on the Dueling Deep Q-Network(Dueling DQN)algorithm,Soft Actor-Critic(SAC)algorithm and Deep Deterministic Policy Gradient(DDPG)algorithm in deep reinforcement learning,this paper studies the obstacle avoidance and tracking problem of UAV.The main contents of this research are as follows.1.Aiming at the obstacle avoidance and tracking problem of UAV in discrete environment,an Improved Dueling DQN(IDueling DQN)algorithm is proposed.Based on the Dueling DQN algorithm,this algorithm combines a new exploration policy and experience pooling mechanism,and endows UAV with the ability of environmental perception.The simulation results show that the improved algorithm has shorter motion path,higher tracking success rate and good generalization.2.Aiming at the obstacle avoidance and tracking problem of UAV in a continuous environment which is perceived by a simulated RGB camera,an Improved SAC(ISAC)algorithm is proposed.Based on SAC algorithm,this algorithm combines the prioritized experience replay mechanism and experience pooling mechanism,and establishes the state space of UAV as a continuous model by using the method of image input.The action space adopts single channel control based on motion direction angle.Meanwhile,a continuous reward function is designed.The simulation results show that the agent using ISAC algorithm has good environmental adaptability.Compared with SAC algorithm and Actor-Critic(AC)algorithm,the improved algorithm has better convergence and higher tracking success rate.3.Aiming at the obstacle avoidance and tracking problem of UAV in the continuous environment which is perceived by a simulated lidar,an Improved DDPG(IDDPG)algorithm is proposed.Based on DDPG algorithm,this algorithm combines delayed-update strategy,adversarial attack method and experience pooling mechanism,and establishes the state space of UAV as a continuous model by using the method of simulating the lidar sensing environment.The action space adopts dual channel control based on motion direction angle and speed.Meanwhile,a continuous reward function is designed.The simulation results show that the agent using IDDPG algorithm has good generalization.Compared with DDPG algorithm and Recurrent Deterministic Policy Gradient(RDPG)algorithm,the improved algorithm has better convergence,higher tracking success rate and stronger anti-interference ability.