Mobility-Aware Trajectory Design Of UAV Base Station Using Reinforcement Learning

In recent years,with the rapid development of unmanned aerial vehicle(UAV)technology,UAVs are being widely used in various communication scenarios,such as aerial relay,data collector of internet of things devices,aerial base station,etc.Compared with the ground communication system or the communication system based on high altitude platforms,a low-altitude UAV equipped with a base station can be deployed and reconfigured more quickly and flexibly.A high mobility of the UAV means a high probability of short-range line-of-sight(LoS)link.Therefore,it is of great importance to study the path planning of aerial base stations.In a dynamic environment considering users’ mobility,we need to solve a thorny problem about how to dynamically adjust the trajectory of an aerial base station in order to provide better communication conditions for users.This thesis is devoted to studying the path planning of an UAV base station in a dynamic environment based on reinforcement learning.The main work and innovations are as follows:(1)A mobility-aware path planning algorithm based on reinforcement learning for an aerial base station is proposed.In this thesis,a novel UAV-assisted emergency communication scenario is proposed,which considers the ground and aerial users with different mobility models.Reinforcement learning algorithms are used to realize the dynamic 2D path planning of the aerial base station to maximize the upload data rate of the system.A simulation environment is coded based on the OpenAI Gym framework.In order to realize the forward-looking intelligent path planning for the aerial base station,an effective neural network structure is designed to fit the Q function of the reinforcement learning algorithm to improve the adaptability of the model under dynamic environment.Simulation results show that compared with the traditional reinforcement learning methods,the performance of this algorithm is greatly improved.The convergence speed is faster.It has a better ability to adapt to the dynamic environment.(2)A 3D path planning algorithm for an aerial base station based on transfer learning is proposed in this thesis.Considering the more general communication scenario,the height of the aerial base station will affect the probability of possessing LoS links and need to avoid obstacles due to the influence of high obstacles.Therefore,a 3D obstacle avoidance path planning problem for an aerial base station is proposed.Compared with the 2D path planning problem in(1),the state-action space of this reinforcement learning problem is more complex and requires a longer exploration and learning process,which is not conducive to solving practical problems.Based on transfer learning,the knowledge acquired by the reinforcement learning algorithm in(1)is transferred to a more complex 3D path planning task,in order to achieve faster convergence of the algorithm.Experiments show that the proposed path planning algorithm has the characteristics of transferable,extensibility and high efficiency.