Multi-user Intelligent Power Allocation Based On Reinforcement Learning

With the development of wireless communication,the phenomenon of low utilization of spectrum resources has aroused widespread concern.In order to make more efficient use of the limited spectrum resources,a number of technologies are used to achieve spectrum sharing.However,the realization of spectrum sharing also brings inevitable problems.Therefore,it is more and more important to reduce the interference between users and realize the reasonable and effective allocation of resources.For a single user,the transmission power of other users who use the same spectrum resource are regarded as the interference.Excessive transmission power will threaten the communication of users.Therefore,it is of great significance to achieve intelligent power allocation through appropriate algorithms.The optimization algorithms are often of high complexity,and it is difficult to meet the real-time requirements under the dynamic environment.This paper will study the multi-user power allocation scheme in cognitive radio based on reinforcement learning algorithm.For the scenario setting of noncooperative power control,there is no interaction between users,and agents can not perceive the global state of the environment.A stateless hypothesis is proposed,and the theoretical framework of multi-agent Q-learning is derived.Because the traditional multi-agent Q-learning algorithm requires the agents know the strategy information of other agents,which cannot be realized under the noncooperative assumption,this paper proposes a multi-agent Q-learning algorithm based on the reward estimated by historical experiences.By taking the historical average of the reward obtained by the corresponding action,the intelligent power allocation is realized without any interaction.Experimental results show that the algorithm can converge to the Nash equilibrium,and the speed of the convergence is faster.In addition,we compare our algorithm with the traditional optimization algorithm.The optimization algorithm can find out how many users can meet their Qo S requirements at the same time in a given scenario,and the algorithm in this paper can achieve this optimal solution in the same scenario.The Nash equilibrium of reinforcement learning algorithm does not necessarily guarantee that all users in the system meet their Qo S requirements,but in the cognitive radio scenario,the Qo S of the primary user should be guaranteed first.Therefore,a new reward function is designed,so that when the user adjusts its power,the Qo S requirement of the primary user should be satisfied first,and then the secondary user improves its own throughput as much as possible.Experiments show that the proposed power allocation scheme can allow more users using the same spectrum resource at the same time and Ensure the fairness of resource allocation.