Research On UAV Communication Traj Ectory Optimization Based On Deep Reinforcement Learning

Due to the advantages of low cost,flexible operation and high safety,UAV(Unmanned Aerial Vehicle,UAV)is widely used in search and rescue,military,agriculture and other fields.The UAV can realize data collection and transmission through the communication system,perform real-time video transmission,and transmit the data to the ground station for processing and analysis.UAV communication research mainly includes path planning,trajectory optimization,resource allocation and resource optimization,etc.,but it is difficult to obtain accurate and processable end-to-end channel models and parameters required for corresponding optimization and optimization;and Even with an accurate channel model and all relevant parameter information,most offline optimization problems are highly non-convex and difficult to solve efficiently.Therefore,in order to overcome this difficulty,this paper adopts Deep Reinforcement Learning(DRL)to solve the optimal decision-making problem of UAVs in complex dynamic environments.A trajectory optimization algorithm based on DRL in the UAV communication scenario is proposed.The specific research work is as follows:First,For the network-connected UAV communication system for cellular connections,an energy-constrained UAV flies from a specific starting point to an end point,while uploading data to Ground base stations(GBS)along the way.In the process,UAVs also need to meet the quality of services(Qo S)requirements,which requires reasonable trajectory design using the controllable mobility of UAVs to avoid the coverage holes of cellular base stations.In this paper,an effective compromise between UAV endurance and system throughput is achieved by jointly designing UAV mission execution time,communication scheduling,UAV trajectory and transmission power.Firstly,the channel environment of the low-altitude network is established and a radio map(Radio Map)is generated,and secondly,based on the radio map,the UAV trajectory optimization problem of the compromise between UAV endurance and system throughput is constructed.In order to solve the established problem,it is transformed into a Markov decision process(Markov Decision Process,MDP),and a multi-step Dueling double deep Q network(Double Deep Q networks,DDQN)algorithm is used to solve the trajectory of the design UAV question.This method abandons the traditional modeling-based solution method.After simulation verification,it proves the effectiveness of the DRL algorithm and successfully solves the trajectory design problem of energy consumption-throughput tradeoff.Second,for UAV data acquisition system,this paper studies a trajectory design strategy based on DRL.In an area where there is a no-fly zone,drones are used to collect data from multiple ground Io T devices in the area.Different from the existing methods of line of sight(Lo S)communication link channel model,this paper adopts a more practical probabilistic Lo S channel model,which considers path loss and shadowing,and satisfies the data throughput of all terrestrial Io T devices.Under the premise of the quantity requirement,by jointly optimizing the trajectory of the UAV and the communication scheduling of the ground equipment,the task completion time can be minimized.To solve the non-convex and hard-to-solve problems,first,the original problem is transformed into a Markov decision process problem,and then a trajectory design solution based on the DRL algorithm is proposed to achieve the goal of time minimization.In the process of executing the algorithm,the drone acts as an agent,interacts with the environment,and continuously improves its own movement strategy.The final simulation results show that the design method proposed in this paper contributes to a significant improvement in system performance and can be applied to real scenarios with no-fly zones.