Resource Allocation And Path Planning For UAV Network Based On Reinforcement Learning

This paper makes a series of research and exploration on the computation offloading strategy of UAV network-assisted mobile edge computing（MEC）.In the single-UAV network and multi-UAV network proposed in this paper,respectively,UAVs are equipped with computing resources to receive the computation offloading request generated by the terminal equipment in a certain area.The terminal equipment divides the generated tasks into two parts.Some tasks are executed locally at the terminal,and the remaining tasks are offloaded to the UAV and receive the processing results returned by the UAV.Under the constraints of space and energy in the assumed UAV network,this paper jointly optimizes the resource allocation and path planning in the UAV network to minimize the maximum processing delay of terminal equipment tasks.Firstly,this paper designs a single-UAV network and models the scenario,considers the communication model and calculation model of the single-UAV network,and then formally describes the problems in the single-UAV network.To solve this problem,this paper adopts single agent reinforcement learning（SARL）algorithms.Firstly,the Q-learning algorithm is adopted,but this algorithm cannot deal with continuous space problems well.Then the Deep Q-network（DQN）algorithm combined with deep learning is adopted,but it is slightly weak in the face of UAVs flying from any angle.Finally,this paper adopts the Deep Deterministic Policy Gradient（DDPG）algorithm.Therefore,these three algorithms solve the problems of a single-UAV network step by step.By synthesizing a variety of algorithms,this paper obtains the optimal task unloading strategy in an uncontrollable dynamic environment.Then,based on the single-UAV network,this paper proposes a multi-UAV network scenario,which is also modeled.Aiming at the joint resource allocation and path planning of multi-UAV network this paper adopts Multi-Agent Reinforcement Learning（MARL）algorithms: Independent DQN algorithm and Multi-Agent Deep Deterministic Policy Gradient（MADDPG）algorithm.MADDPG algorithm can deal with the problems of joint action,continuity of state space,continuity of state space,and so on.The experimental results show that the MADDPG algorithm can converge quickly.At the same time,compared with the baseline and IDQN algorithm,the MADDPG algorithm can minimize the task processing delay as much as possible.In the experimental verification part,a series of experiments are carried out on the two UAV networks proposed in this paper and compared with the baseline,which proves that the DDPG algorithm and the MADDPG algorithm adopted in this paper have good results.The above systematic analysis and experiments show that the resource allocation and path planning of UAV networks based on reinforcement learning can achieve good performance in the current related research,and provide a direction for the development of the next generation communication system.