Research On Uplink Power Control For Multi-cell NOMA System Based On Reinforcement Learning

With the rapid development of communication networks,the number of users has increased quickly.The number of terminal equipment supported by the existing orthogonal multiple access technology,is limited by the number of orthogonal resource blocks,and thus it is difficult to meet the massive access scenarios in future communication systems.In order to improve the transmission rate and meet the demand of massive access,there is an urgent need to develop new multiple access solutions.As a key technology for future communication systems,non-orthogonal multiple access(NOMA)is capable of enhancing network capacity of communication system substantially and thus has received extensive attention from the scholars at home and abroad.In the multi-cell NOMA system,the interference becomes extremely complicated,including inter-cell interference and inter-NOMA user interference.Therefore,in order to ensure the quality of service for each user,the design of effective power control schemes becomes very essential.Meanwhile,reinforcement learning,treated as an important research direction in the field of machine learning,has attracted the focus of researchers in the recent years.Currently,it has become common to leverage reinforcement learning methods to solve problems in communication systems.Based on reinforcement learning,this paper considers two types of scenarios with discrete and continuous transmission power spaces,respectively.Besides,the power control scheme is studied for uplink multi-cell NOMA system from the perspectives of single-agent centralized control and multi-agent distributed cooperative control.The research outputs are listed as follows:Firstly,a centralized power control scheme based on single agent learning algorithm is proposed to solve the power control problem of NOMA uplinks with the discrete spaces.In order to resist the influence of path loss in the considered system model,and effectively distinguish the user signals at the base station,a fractional power control(FPC)method with multiple-path compensation loss is proposed,which effectively solves the shortcomings of the existing power control scheme with single-path loss compensation factor.At the same time,in order to maximize the system sum rate under the premise of satisfying the minimum transmission rate of each user,this paper proposes to use a Q-learning algorithm to jointly optimize the path loss compensation factors of the near and the far uses of NOMA system,which simultaneously takes into account inter-cell interference and inter-NOMA user interference.Simulation results show that the proposed power control scheme in this paper can achieve higher sum rate performance than the existing schemes.Secondly,for the uplink power control problem for multi-cell NOMA with continuous space,a distributed cooperative power control scheme based on multi-agent learning algorithm is proposed.The user’s power space in the considered system model is defined as a continuous set.Considering the limitation of a single agent’s ability to process data,and inspired by the concept of distributed artificial intelligence,this paper proposes to use a multi-agent deep deterministic policy gradient(MADDPG)algorithm to solve the user power control problem.In the proposed scheme,each user is treated as an independent agent,and the Actor-Critic framework is used to ensure the information interaction between agents and update the behavior strategy.In simulations,the influence of the agent update parameters on the learning algorithm is studied,and the proposed scheme is compared with the single-agent deep deterministic policy gradient(DDPG)algorithm and the discrete power control scheme proposed in Chapter 3.It is shown that the power control scheme based on MADDPG algorithm can obtain better system sum rate performance.