Research On Resource Allocation Algorithm For Ultra-Dense Small Cell Networks Based On Deep Reinforcement Learning

With the rapid development of mobile communication technology and the popularization of mobile devices,there is an increasing demand for high-speed,stable,and reliable mobile communication.Ultra-dense small cell networks(UDSCN)have emerged as a new form of wireless communication network,with advantages such as flexible deployment,high spectrum utilization,and low energy consumption,making it an important means to enhance network capacity and coverage.However,due to its high base station density and complex channel environment,resource allocation in UDSCN has become particularly complex and challenging.Deep reinforcement learning,as an emerging machine learning method,can automatically learn the optimal strategy from historical data without requiring an accurate problem model.Applying deep reinforcement learning to resource allocation in UDSCN has important research significance.This thesis proposes resource allocation algorithms for UDSCN based on deep reinforcement learning,starting from three aspects:power allocation,joint resource allocation,and resource allocation framework,providing new ideas and methods for solving the resource allocation problem in UDSCN.Firstly,an A2C(Advantage Actor-Critic)based UDSCN power allocation algorithm(AUPA)is proposed to maximize the system rate of UDSCN by optimizing the power allocation strategy.The system model of small cell networks is first analyzed,and the power allocation problem is modeled.Then,detailed mathematical derivation of the policy gradient is carried out,and the algorithm design and flow for small cell power allocation are given.Finally,through simulation experiments,the AUPA algorithm can adapt to complex network scenarios and effectively improve the overall transmission rate of the network.To further optimize the resource allocation effect of UDSCN,a PPO(Proximal Policy Optimization)based UDSCN resource allocation algorithm(PURA)is proposed,which integrates subcarrier allocation and power allocation to maximize the system energy efficiency while considering the quality of service(QoS)requirements of user devices.Simulation results show that the PURA algorithm can effectively reduce the overall energy consumption of the network while maximizing the system rate and improving the QoS satisfaction of users.Finally,an O-RAN(Open Radio Access Network)based UDSCN resource allocation framework(O-URAF)is proposed to provide a highly feasible solution for.the application of deep reinforcement learning algorithms in UDSCN.O-URAF includes an execution module and a training module.Through modular design,the execution module can obtain the optimal resource allocation strategy in real-time,while the training module can train the neural network using global channel state information.Simulation results show that O-URAF can improve the performance of the resource allocation algorithm with extremely low execution delay and has significant practical application value.