Optimization Of Flight Trajectory In UAV Communication System Based On Reinforcement Learning

In recent years,unmanned aerial vehicles(UAVs),with their high mobility,low cost,and easy deployment,have become one of the key enabling technologies in the wireless communication field.However,the problem of insufficient energy supply for UAVs poses many challenges for their application in communication systems.The total energy consumption of UAVs is mainly composed of flight propulsion energy consumption,which is determined by the UAV’s trajectory.Therefore,trajectory design directly affects the endurance of UAVs and the operational efficiency of the entire UAV communication system.This thesis studies the optimization problem of flight trajectory in UAV communication systems based on reinforcement learning algorithms.System models are established to meet the requirements of practical application scenarios,and optimization schemes are proposed to improve system performance.The main innovative work of this thesis is as follows:1.A novel optimization scheme for single unmanned aerial vehicle data collection flight trajectories is proposed.In the construction of the system model,the UAV collects data stored in ground Internet of Things(IoT)devices with unknown location information,and the objective of the system is to minimize the completion time of data collection tasks.To achieve this goal,this paper presents an algorithm based on deep Qnetwork(DQN)to optimize the flight trajectory of the UAV.The algorithm overcomes the limitations of existing algorithms that are only applicable to a single scenario,and does not require repeated training when the scenario changes.Simulation results demonstrate that under the condition of unknown ground device location information,the proposed algorithm achieves an efficiency of 96.1%compared to the benchmark DQN algorithm with known ground device precise locations,validating the effectiveness of the proposed trajectory optimization scheme.2.A multi-UAV base station flight trajectory optimization scheme is proposed.In the constructed system model,multiple UAVs assist ground terminal user communication based on a probabilistic line-of-sight model,and the objective of the system is to maximize the total uplink link rate while ensuring user fairness.To achieve this objective,this paper formulates the trajectory optimization problem as a Markov decision process and proposes an algorithm based on deep deterministic policy gradient(DDPG)to optimize the flight trajectories of multiple UAVs.Simulation results demonstrate that compared to other algorithms,the proposed algorithm achieves higher total uplink link rate and user fairness index.