Deep Reinforcement Learning-based Resource Allocation Algorithm Research For Heterogeneous Cloud Access Network

In order to meet the high-speed,high-reliability,and low-latency requirements of 5G networks,heterogeneous cloud wireless access network(H-CRAN)is an inevitable trend in the future development of 5G networks.Through resource virtualization and centralized allocation,network spectrum efficiency and energy efficiency are further improved to achieve higher throughput performance.However,the dynamic characteristics of wireless resources and the complexity of the wireless network environment bring great challenges to resource optimization strategies.Network self-optimization is the key to achieving reasonable,rapid,and on-demand resource allocation.This thesis studies the resource allocation of heterogeneous cloud line access networks based on deep reinforcement learning theory.The main research contents and innovations are as follows:1.In order to meet the demand for wireless data traffic to greatly increase and realize the dynamic allocation of virtual resources,a wireless resource allocation algorithm based on deep reinforcement learning(DRL)is proposed.Firstly,a stochastic optimization model for maximizing the total network throughput was established to jointly optimize the congestion control,the user association,subcarrier allocation and the power allocation under the constraint of queue stability.Secondly,considering the complexity of scheduling problem,the state space and action space of the system have high-dimensional features.The DRL algorithm uses neural network as nonlinear approximate function to solve the dimensional disaster problem efficiently.Finally,aiming at considering the complexity and variability of the wireless network environment,the transfer learning(TL)algorithm is introduced to make use of the small sample learning characteristics of TL so that the DRL algorithm can obtain the optimal resource allocation strategy in the case of insufficient samples.In addition,TL further accelerates the convergence rate of DRL algorithm by transferring the weight parameters of DRL model.Simulation results show that the proposed algorithm can effectively increase network throughput and improve network stability.2.Aiming at the problems of spectrum efficiency and energy efficiency of heterogeneous cloud wireless access networks,a PD-NOMA-based energy efficiency optimization algorithm was proposed.First,the algorithm constrains queue stability and fronthaul link capacity as constraints,jointly optimizes user association,power allocation,and resource block allocation,and establishes a joint optimization model for network energy efficiency and user fairness.Secondly,because the state space and action space of the system are both high-dimensional and continuous,the research problem is a continuous domain NP-hard problem,and then a trust region policy optimization(TRPO)algorithm is introduced to efficiently solve high-level problems.Dimensional motion space and continuous domain problems.Finally,the standard calculation method for the TRPO algorithm is too large.Proximal policy optimization(PPO)algorithm is used to optimize the solution.The PPO algorithm not only ensures the reliability of the TRPO algorithm,but also effectively reduces the TRPO algorithm computational complexity.Simulation results show that the proposed algorithm improves the network's energy efficiency performance under the constraint of ensuring user fairness.