Research On Spectrum Allocation Method Of Cognitive Radio Network Based On Reinforcement Learning

With the rapid development of the mobile Internet and the continuous updating of smart terminal technology,the number of wireless mobile users has been increasing in the past few years.This trend is expected to continue in the coming years.Considering the vigorous development of the number of users,the mobile network's mobile traffic will also continue to increase.To meet the future needs of mobile communications,the network capacit y should be continuously improved in the future.An effective way to increase capacity is to allocate more spectrum resources to wireless communication systems.However,due to the rapid development of the spectrum-based industry,frequency bands have become overcrowded,but users' demand for spectrum resources has become more and more urgent.Therefore,the spectrum has become a scarce resource,so it is unrealistic to allocate sufficient spectrum resources to each user in the network.On the other hand,considering that the traditional spectrum management strategy still has some shortcomings,for example,only authorized primary users are allowed to use the frequency band.Unauthorized secondary users are not allowed to use the frequency band.Such problems cause the allocated spectrum resources not to be fully utilized.Based on this background,it is urgent to improve the utilization rate of the frequency spectrum,and thus cognitive radio technology is born.Cognitive radio technology is based on cognitive ability,and it can learn from the environment and adapt to the environment,which provides an effective solution to alleviate the problem of spectrum shortage and spectrum utilization.It provides effective solutions to alleviate the problems of spectrum shortage and spectrum utilization.In cognitive radio networks,reinforcement learning algorithms with autonomous learning capabilities can better solve the problem of spectrum allocation.Reinforcement learning algorithms can determine the optimal strategy for a finite Markov decision process(MDP),which is used to model decision-making for dynamic spectrum access problems under uncertainty.Q-Learning is an effective reinforcement learning decision model.Therefore,the dynamic spectrum access technology based on Q-Learning has extremely important research significance.Based on this background,the specific research contents of this paper are as follows:(1)This paper proposes an improved Qo E-driven Qo E-driven cognitive radio network spectrum resource allocation algorithm.A mathematical model is constructed from the perspective of Q-Learning,with multiple auxiliary users in the cognitive radio network as multiple subject learners without prior knowledge of mutual interference.We redesign the function of the reward function,the basic principle of which is to give strong rewards for continuous correct behaviors and strong punishment for continuous wrong behaviors during the learning process.The improved reward function can better stimulate the learning potential of the algorithm,while being closer to the actual situation and more intelligent and user-friendly.In addition,the average opinion score(MOS)has become a widely used indicator of the quality of end-users' subjective experience.Therefore,we have developed a distributed dynamic spectrum access(DSA)solution based on MOS,which can coexist between the PU and the SU under the condition that the primary user(PU)interference constraint is met and the total MOS value is the largest.Simulation results show that in most cases,the proposed algorithm is superior to the traditional Q-Learning algorithm in terms of MOS value and average bit rate,while ensuring a good quality of experience for users.(2)Based on the original Q-Learning and game theory,a Q-Learning based on game theory was proposed.A mathematical model is constructed from the Q-Learning perspective of joint games,and many secondary users in cognitive radio networks are considered learners.We only need to take advantage of its historical state without having to know the prior knowledge of interference between users.By introducing Q-Learning,the primary and secondary users can be allowed to share the frequency spectrum in a fair manner while meeting the tolerable interference constraints of the primary user.Considering that the traditional Q-Learning algorithm is inefficient,it cannot obtain better spectrum resource allocation.To adopt a better method in spectrum resource allocation,game theory and Q-Learning algorithms are combined to complete decision-making.When studying the behavioral collaboration of multiple learners,an improved Pareto Q-Learning algorithm is proposed based on the benefits of cooperative alliances.The algorithm is based on multiple learner cooperative alliance theory and Markov game theory.The global target is regarded as the target of reinforcement learning as the local Pareto optimal joint behavior.At the same time,a common benefit based on acceptability distribution is proposed to transform the optimal behavior in the global sense into local Pareto behavior through iterative learning.Simulation results show that our proposed method is superior to the original Q-Learning in terms of system throughput and system collision rate.Therefore,this proves that game-based Q-Learning is feasible and effective.