Research Of Frequency Hopping Spectrum Sensing Based On Reinforcement Learning

Under the background of the rapid development of information and communication technology,problems such as the shortage of spectrum resources and the low utilization rate of spectrum have a great impact on the demand for spectrum resources by the ever-increasing users.Cognitive Radio(CR)technology is a radio technology used to solve the shortage of spectrum resources and improve spectrum utilization.It has the ability to actively observe the spectrum conditions in the space and act as a primary user(PU)of a frequency band.When the frequency band it occupies is temporarily not used,this idle frequency band can be handed over to Secondary Users(SU)for transmission.This paper mainly uses the basic physical characteristics of Frequency Hopping Spread Spectrum(FHSS)to construct the core technology of cognitive radio-spectrum sensing,which constitutes the idea of frequency hopping spectrum sensing,and studies its use in spectrum reuse.Application value,and finally use reinforcement learning to optimize frequency hopping spectrum perception.The main research content and innovations of this article can be divided into the following three parts:(1)By analyzing and comparing the shortcomings of traditional energy sensing and the degradation of its sensing ability in a complex spectrum environment,this paper proposes a threshold detection technology based on traditional energy sensing,which uses the frequency hopping sequence transmitted by the user as the main sensing basis and passes the signal The instantaneous energy completes the frequency hopping spectrum sensing idea of spectrum sensing.By building a simulation model to simulate frequency hopping spectrum sensing,frequency hopping spectrum sensing,has the ability to perceive the noise environment in the current frequency band.At the same time,under the condition of low signal-to-noise ratio,the sensing ability of frequency hopping spectrum sensing is 10%compared with traditional energy sensing.Upgrading from left to right.(2)Based on the idea of the FHSS system,this paper proposes a spectrum reuse idea in which PU signals and SU signals are connected to a fixed frequency band at the same time.Through simulation,the traditional energy perception and frequency hopping spectrum perception are compared under this spectrum reuse idea.As a result,using the unmixing ratio and spectrum reuse rate as the measurement indicators,it is concluded that frequency hopping spectrum sensing is more suitable for formulating multiplexing strategies for the spectrum multiplexing mode where two signals are simultaneously connected to a fixed frequency band.At the same time,this article also designed two improved schemes for frequency hopping spectrum sensing:"forward detection" and "backward prediction".Experiments show that both improved schemes can effectively improve the efficiency of spectrum reuse.And the improved "backward detection" scheme is more suitable for the formulation of spectrum reuse strategies for frequency hopping spectrum sensing.(3)This paper proposes the frequency hopping spectrum sensing of reinforcement learning,and uses reinforcement learning to solve the problem of effective signal reception interruption.This paper builds a reinforcement learning model based on frequency hopping spectrum sensing,and uses QLearning and gradient strategy(Policy Gradient,PG)algorithms for training.Through training,it is found that using the average reward as a measurement standard,Q-learning can quickly achieve the average reward convergence through the Q value update,but as the number of rounds increases,the Q value update appears to be over-fitting,and the average reward declines;while PG does Convergence is slow,but according to the method of updating the strategy of the loss function,there will be no overfitting,and the average reward always maintains convergence.Based on the above three studies,this paper constructs a frequency hopping spectrum sensing model based on reinforcement learning,from the signal receiving and using reinforcement learning to analyze,to determining the multiplexing strategy after the main user signal,and improving the spectrum through spectrum sensing ideas.Utilization rate effectively solves the problem of shortage of spectrum resources.