Research On Resource Scheduling Technology Based On Deep Reinforcement Learning In Hetnets

With the increasing number of mobile users,the next generation of wireless networks needs to support higher user density than the current network.One way to meet this need is to share network resources more efficiently through femtocells.Femtocells are a key component of the Long-Term-Evolution-Advanced(LTE-A)version that has been increasingly used in recent years,providing higher system throughput and higher frequency utilization at lower power requirements.However,the main challenge in deploying these networks effectively is how to schedule network resources.Due to the lack of fair and reasonable resource allocation strategy for users,the more complex interference environment caused by the complex deployment of heterogeneous networks(HetNets)including macro cells and femtocells will seriously affect the system performance,especially the data transmission rate and overall system performance of users at the edge of the cell.Therefore,in order to alleviate the problems caused by these disturbances,effective interference coordination and resource scheduling scheme are needed.Power is a key component of network resources,and the power allocation scheme of base station to users directly affect the communication quality of users.Although traditional power allocation method can effectively improve the communication quality of users,unpanned deployment of femtocells requires an adaptable and self-organizing algorithm to make HetNets viable.In order to build a self-organizing dense femtocell HetNet,this paper applies Reinforcement Learning(RL)to the power distribution of densely deployed femtocell network and puts forward a power allocation scheme based on reinforcement learning in HetNets,maximizing the total network capacity while providing Quality of Service(QoS)and fairness to all users.In this paper the power allocation process in HetNets is modeled as Markov decision process,the dense HetNet is modeled as a multi-agent network,each of the stations in the HetNets are considered to be an agent.Agent interacts with the environment,the transmission power of base station is modeled as the action space of an agent,and agent get feedback through reasonable design of reward function.As for Q-learning cannot handle infinite agent state space effectively and quickly,the deep Q network(DQN)is used to enable the agent take system capacity and fairness of users into account when performing action selection with using neural network with powerful feature extraction ability.The power allocation scheme of base station makes the total capacity of the system move in the direction of improvement.By integrating RL method and deep learning into wireless communication,a self-organized and adaptive dense femtocell network is constructed to effectively schedule resources in HetNets.The simulation results show the proposed method can effectively improve the system capacity compared with previous methods.