Research On UAV Trajectory Planning Based On Deep Reinforcement Learning

The widespread use of Unmanned Aerial Vehicles(UAVs)in the conflict between Russia and Ukraine is pushing for updates and iterations in the military combat field.The trajectory planning is a key part of UAV autonomous flight control,which can provide safe and reliable flight trajectories for UAVs to accomplish complex tasks on the battlefield.With the rapidly changing battlefield situation,traditional UAV trajectory planning algorithm’s capabilities are weak and cannot meet the autonomous and precise strike requirements of modern electronic warfare.This article focuses on research on autonomous trajectory planning for UAVs in dynamic and complex combat environments based on technologies such as environment modeling,Deep Reinforcement Learning(DRL),and improved DRL.The main research contents are as follows.Firstly,an UAV flight environment is constructed,the threat elements in the environment are analyzed,trajectory planning is integrated with DRL,and the key elements of trajectory planning DRL are designed.Following that,the Actor-Critic framework is introduced,and a Deep Deterministic Policy Gradient(DDPG)-based UAV trajectory planning algorithm is proposed.The correctness of the algorithm is verified by simulation.Secondly,the improvement of key elements of DRL for UAV path planning is studied.The gated neural unit is fused to improve the Actor-Critic network structure,and the UAV path planning algorithm based on space-time perception is proposed.The stability of UAV action decision in training is verified by simulation.On this basis,the absorption terminal reward are designed in conjunction with Markov state transition,the empirical array of absorption terminal state are composed to optimize the experience replay pool.Following that,an absorbing terminal-based UAV trajectory planning algorithm is proposed.The feasibility of the algorithm to reduce the training time is verified by simulation.Finally,the fusion of DRL and Generative Adversarial Imitation Learning(GAIL)is studied,and the DRL reward mechanism is further optimized.Experience generation and highquality selection standards are set,and afterwards,a GAIL-GDDPG-ABS-based UAV trajectory planning algorithm is proposed.On this basis,according to the steady-state preservation principle,high quality absorption terminal state experience are labeled.An absorption terminal discriminator is designed and the discriminant evaluation is optimized.Morever,a GAIL-GDDPG-2ABS-based UAV trajectory planning algorithm is proposed.The autonomy and reliability of the proposed algorithm for dynamic complex environments are verified by simulation,which pave the way for advancing artificial intelligence in the field of UAV autonomous flight control.