Research On 6G Multi-tier Network Resource Allocation Based On Deep Reinforcement Learning

In the 6G era which network trends to be ultra-dense,so serious interference and complex network optimization are a major problem faced by 6G.In addition,the structure of6 G communication network will also be larger and more heterogeneous,and the types of services and application scenarios will also be more complex and changeable.Bacause using artificial intelligence technology to intelligently manage network resources and user equipment is an effective way to improve network efficiency,this thesis studies the resource allocation problem in 6G multi-tier cellular network based on deep reinforcement learning(DRL).The specific work is as follows:First,this thesis considers a two-tier heterogeneous cellular network consisting of pico base station and femto base station.In order to simulate real-life scenarios more accurately,the base stations are modeled as a Poisson point process,and users are randomly distributed in the network.In order to improve the transmission rate of the network,a DRL-based downlink power allocation scheme is proposed.In this network model,a deep Q network(DQN)with two hidden layers is constructed,and the network parameters are continuously optimized to maximize the network transmission rate.Simulation results show that the proposed scheme is superior to other algorithms in terms of transmission rate and convergence speed.Secondly,in order to explore the influence of key network parameters on the performance of deep Q learning(DQL),a two-tier heterogeneous cellular network with the ultra-dense deployment of small cells is considered.In this network model,the ultradense deployment of small cells will lead to serious interference and increase the energy consumption of the network.In order to improve the energy efficiency of the network,a power control scheme based on DQL is proposed.This scheme reduces the interference in the network by regulating the transmitting power of the base station,and maximizes the network energy efficiency under the premise of ensuring the user equipment quality of service requirements.In addition,in order to explore the influence of key network parameters on the performance of the scheme proposed in this thesis,the effects of different discount factors,the number of small cells and the number of user equipment in each small area on the energy efficiency of the network are compared and analyzed.Simulation results show that the proposed scheme is superior to other algorithms in terms of energy efficiency.Finally,this thesis considers a three-tier heterogeneous cellular network composed of micro base stations,pico base station and femto base station,and studies the joint optimization of user association and resource allocation in the network.Since this joint optimization problem is non-convex and NP-hard,and in order to further improve the performance of the network,a joint optimization scheme based on double DQN(D-DQN)is proposed.In this network,the spectral efficiency is taken as the reward and penalty value,and the weight of D-DQN is continuously trained by using the batch gradient descent method to maximize the spectral efficiency of the network.In addition,the effects of different reinforcement learning methods on network transmission rate and energy efficiency are simulated and analyzed.Simulation results show that the performance of D-DQN proposed in this thesis is better than other reinforcement learning schemes.