Research And Implementation Of Unmanned Aerial Vehicle(UAV) Autonomous Flight Control Algorithm Based On Deep Reinforcement Learning

With the development of Unmanned Aerial Vehicle(UAV)technology,UAV has been widely used in many fields.At the same time,the requirements for the function and performance of UAV are increasingly stringent.There are many key basic and common technologies to be solved,among which autonomous flight control is the most basic and important technology.According to the characteristics of UAV autonomous flight control mission,this paper selects Deep Deterministic Policy Gradient(DDPG)as the basic algorithm for research.Aiming at the sharp increase of state space caused by UAV long-distance target mission.To solve the problem of sparse reward in UAV autonomous control mission and help agents learn the strategy of completing tasks in sparse reward scene more quickly and effectively,a reinforcement learning algorithm based on Sub-Goal Generation and Empirical Feature Storage(SGG-EFS)is proposed.The main work and contributions are as follows:(1)A reinforcement learning algorithm based on Empirical Feature Storage(EFS)is proposed.The feature value of experience is extracted by feature extraction method to control the storage of experience,which is convenient for the management of effective experience in the experience pool and ensures that as many different experiences as possible are stored for agent learning.This method avoids the problem that the network can not fully learn the experience obtained because a large number of repeated experiences occupy the experience pool space.Eigenvalue records are accessed and queried by hash,which can improve the query speed.(2)A Sub-Goal Generation(SGG)algorithm is proposed to split long-distance target tasks.After splitting different task scenes,it can alleviate the growth of state space to a certain extent,and even produce similar sub target scenes.The experience learned by the agent before can be directly used in new similar scenes,which can improve the generalization ability of the algorithm model.(3)Compared with the DDPG algorithm and Priority Experience Replay(PER)algorithm,the SGG-EFS algorithm has better performance in learning speed and strategy stability.In order to verify the effectiveness of the algorithm,the SGG-EFS algorithm is applied to the shipboard aircraft autonomous control task in the go around scene of X-Plane simulator,and the Markov decision process model is designed for the shipboard aircraft autonomous control task.The SGG-EFS algorithm is used to complete the autonomous control task of shipboard aircraft in different scenarios,which proves that the algorithm has a good application effect in practical problems.