Research Of Spectrum Sensing Strategy Based On Reinforcement Learning

Cognitive radio technology is dedicated to changing the fixed spectrum allocation and access method,that is,secondary users can access wireless spectrum resources opportunistically without causing interference to the primary user to improve spectrum utilization.Among them,spectrum sensing technology is the basis of cognitive radio technology,and it is also the focus of research at home and abroad.In spectrum sensing,before the secondary user accesses the channel,in order to minimize the interference to the primary user's communication,the secondary user needs to select the channel in sequence to find the idle channel,which will cause scanning overhead and access delay.For this problem,this thesis relies on the team's scientific research project and aim at assisting secondary users to quickly and accurately find idle channels.This thesis researches from the two aspects of channel selection algorithm and cooperative spectrum sensing algorithm,and carries out the following work:(1)The principles and application difficulties of reinforcement learning models are summarized.Among them,the model of multi-arm bandit problem,action value estimation strategy,and action selection strategy are mainly studied,which lays a theoretical foundation for subsequent research in this paper.(2)In order to speed up secondary users to find idle channels to reduce scanning overhead and access delay,this thesis designs a channel selection model based on reinforcement learning,which models the secondary user channel selection problem as a multi-arm bandit model in reinforcement learning.Existing research assumes that the channel occupation mode of the primary user is an ideal Bernoulli process,and assumes that the channel is perfectly detected by the secondary user.In order to make the model closer to the real environment,the primary user occupancy model is modeled as an associated Bernoulli process,and the detection accuracy of the neighbors are taken into consideration when designing the channel return function of the model.(3)To solve the channel selection model in(2),a FEAC channel selection strategy is proposed,and the performance of the FEAC strategy is studied from both theoretical and simulation perspectives.In theory,the FEAC strategy can fully explore each channel at the initial stage of channel selection,to obtain the actual idle probability of each channel as much as possible.In the middle and late stages of channel selection,it can quickly converge to select the channel with the highest idle probability.In terms of simulation,the proposed FEAC strategy can indeed accelerate the convergence in the solution process and produce fewer regret values.The results of solving the model in(2)with the FEAC strategy show that the channel selection model can effectively reduce scanning overhead and access delay.(4)A collaborative spectrum sensing algorithm based on reinforcement learning is designed for the environment that supports cooperation among users.The cooperative user selection problem is modeled as a multi-arm bandit model.Based on the channel selection algorithm,the secondary user learns the reputation value,that is,the credibility of the detection,of the cooperative user,and selects the cooperative user with the highest reputation value to perceive the channel.Simulation results show that compared with non-cooperative sensing and traditional cooperative spectrum sensing algorithms,the overall detection efficiency of the network is improved,thereby further reducing scanning overhead and access delay.