Reinforcement Learning Based Energy Efficiency Maximization For Downlink NOMA Systems

With the development of mobile communication technology,the number of network equipments has been increased rapidly,and the demand for mobile data traffic has exploded quickly.Therefore,the traditional multiple access methods face great challenges.In such situation,it is an interesting problem that how to improve system performance and support large-scale device access by making full use of limited spectrum resources.In order to solve the above issue,non-orthogonal multiple access(NOMA)technology has been developed.By using the new power domain,NOMA can support multiple users’ signals in a single time-frequency domain resource.Besides,it has also attracted the attention of many researchers because of the excellent performance in improving the number of users and the utilization of frequency spectrum.In addition,as a new innovation technology in wireless communication system,intelligent reflecting surface(IRS)has received much attention.By integrating a large number of passive reflecting elements,IRS can reconfigure wireless propagation environment.IRS can further release the potential of NOMA technology and further improve the performance of NOMA systems,which is of great significance of enhancing network performance.On the other hand,with the popularity of green environmental protection,energy efficiency(EE)has become one of the key issues currently.Therefore,this paper studies energy efficiency optimization of NOMA system and IRS-NOMA system based on reinforcement learning algorithm for energy-efficient application scenarios in future network development.Firstly,an optimized resource allocation scheme is proposed to maximize the energy efficiency for downlink multi-user NOMA system.For energy efficiency optimization of downlink multi-user NOMA system,we divide the non-convex NP-hard primary problem into two sub-problems,i.e.,user grouping and power allocation,which are respectively addressed by the designed schemes.Specifically,a user grouping algorithm based on reinforcement learning algorithm is proposed.Then,under the given channel power allocation and user grouping schemes,a closed-form expression of power allocation for the users in each subchannel is derived.In order to further improve the energy efficiency of the NOMA system,a fractional transmit power allocation(FTPA)is invoked to solve the power allocation across the different subchannels.Furthermore,we also provide a joint optimization algorithm of resource allocation.Simulation results show that,compared with other user grouping methods,the proposed scheme can achieve a higher system energy efficiency.Secondly,a joint optimization scheme of power allocation and phase shift is proposed for maximizing the energy efficiency of downlink IRS-aided NOMA system.For the downlink IRS-assisted NOMA system,this paper develops a energy efficiency maximization problem under the constraints of the minimum rate requirement of each user and the maximum transmitting power of base station.Since it is hard to solve the original problem,it is decomposed into two sub-problems,which are solved separately by using the alternative optimization.Particularly,for given phase shifts,a power allocation algorithm based on DQN algorithm is proposed.Then,we use block coordinate ascent(BCA)technique to optimize IRS’s phase shifts under given power allocation coefficients.Besides,a convergent alternate optimization algorithm is presented.Via simulations,it is shown that the proposed power allocation and phase shift optimization scheme is capable of improving the energy efficiency substantially.