Research On Cognitive Radio Spectrum Access Algorithm Based On Reinforcement Learning

In recent years,the Internet of Things technology has achieved rapid development.The Internet of Things technology has set off a new digital revolution because of its potential ability to combine the physical world with the digital world.One of the main challenges that hinder the vision of the Internet of Things is that the radio spectrum resources suitable for information transmission are insufficient,and cognitive radio technology is considered to be a practical implementation plan to alleviate the tension of spectrum resources.In cognitive radio technology,dynamic spectrum access technology is widely regarded as a core technology,which enables secondary users to detect the presence or absence of primary users in the network by sensing the radio frequency spectrum,and has the opportunity to access unoccupied Spectrum to communicate.The work direction of this paper is to apply the theory of deep reinforcement learning to the dynamic spectrum access algorithm model,and conduct in-depth research on it to construct an algorithm model that can adapt to the current Internet of Things environment.The specific research content is as follows:Firstly,this paper applies two representative value optimization algorithms in reinforcement learning to the dynamic spectrum access algorithm model in the Internet of Things environment,and proposes a dynamic spectrum access algorithm based on the deep Q network and a double deep Q network based Dynamic spectrum access algorithm.The simulation results show that the proposed algorithm has significant effects on the reduction of collision probability between different users and the improvement of system frequency band utilization,which confirms the effectiveness of the proposed algorithm.Secondly,the dynamic spectrum access algorithm based on the deep Q network uses a single "Q value" to represent the state value and action value during the training process,and the value evaluation problem is analyzed.This article adjusts the network structure.The original single "Q value" output is adjusted to the state value function and the action advantage function to output separately.It also analyzes the convergence speed of the dynamic spectrum access algorithm based on Double deep Q network due to the increase in structural complexity.This article changes the method based on random sampling in the original algorithm to the method based on optimized sampling.Samples with different importance will be sampled differently.The simulation results show that the two adjusted algorithms have a more accurate "Q value" output and a shorter convergence period,which confirms the effectiveness of the proposed improved algorithm.Finally,in view of the complex types and different structures of secondary users in the application scenarios of the Internet of Things,it is pointed out that the two improved algorithms proposed in this paper still have the problem of being limited to a single spectrum access method.The improved dynamic spectrum access algorithm is extended from a single spectrum access method to a hybrid spectrum access method,making it more suitable for application in the Internet of Things environment.The simulation results show that the expansion of the hybrid spectrum access method further effectively reduces the number of access conflicts between users and improves the system frequency band utilization,which confirms the effectiveness of the expansion algorithm.