Research On Multi-agent Deep Reinforcement Learning Based Resource Allocation For NOMA Systems

With the development of mobile communication technology,the access of users in the mobile communication network is exploding,and the future mobile communication system needs communication with high transmission rate,high spectrum efficiency and lower delay.Non-orthogonal multiple access technology(NOMA)can increase the data rate,reduce delay,save energy,reduce cost and increase the system capacity when spectrum resources are extremely scarce.Therefore,NOMA can meet the development needs of the future mobile communication system.However,whether the advantages of NOMA technology can be brought into full play depends on if there is an effective way of resource allocation.In this paper,aiming at the uplink and downlink models of NOMA system,the resource allocation methods are studied to optimize the energy efficiency and spectrum efficiency of the system respectively.The main work of this paper is as follows:Firstly,this paper studies the problem of joint subcarrier allocation and power allocation in uplink multi-cell NOMA system,hoping to maximize the energy efficiency while ensuring the minimum data rate of all users.Aiming at this dynamic optimization problem,a multi-agent deep reinforcement learning(MADRL)method with centralized training and distributed execution is proposed.This method not only avoids the disadvantage that the state space and action space in centralized method will increase exponentially with the number of agents,but also takes into account the serious interference and low efficiency caused by independent learning of agents in distributed method.In this method,each cell is regarded as an agent,and a subcarrier allocation network and a power allocation network are designed for each cell.Finally,the whole resource allocation strategy is updated according to the reward of system feedback.Simulation results show that this method can achieve higher energy efficiency compared with the centralized method and the distributed method.Secondly,this paper studies the cooperative clustering and power allocation in the downlink Coordinated Multi-Point-NOMA(Co MP-NOMA)system,hoping to maximize the spectrum efficiency.To solve this dynamic optimization problem,we propose two methods based on MADRL: user-centered allocation method and cell-centered allocation method.The final cooperative clustering scheme of the system is determined in two different ways.In addition,we designs a power class division method,which can determine the output power range of users according to the decoding order of users,and get the final power allocation scheme with the help of MDARL method.The simulation results show that the user-centered allocation method can effectively improve the spectrum efficiency of marginal users,but it sacrifices the spectrum efficiency of central users,and the cell-centered allocation method can properly improve the spectrum efficiency of marginal users and ensure the spectrum efficiency of the whole system.