Research On Cognitive Radio Network Access Technology Based On Reinforcement Learning

Cognitive radio technology breaks the limitations of the current fixed spectrum allocation scheme.And it allows secondary users to adaptively adjust transmission parameters to utilize idle spectrum resources for data transmission,greatly improving spectrum utilization.Benefiting from the learning and reasoning capabilities of massive communication data,artificial intelligence and machine learning methods can dynamically adjust the transmission protocol of the system to adapt to the complex communication and network environment,which represents the development direction of future communication.Therefore,using machine learning to solve problems in cognitive radio has become a current research hotspot.This dissertation focuses on the application of reinforcement learning in spectrum sensing and spectrum access.The main contributions of this dissertation are listed as follows:(1)The development process and current research status of cognitive radio technology are summarized,and then the key technologies of spectrum sensing and spectrum access are introduced.Among then,the basic model and principle of reinforcement learning are summarized.And then the application conditions of the Q-learning algorithm,the update formula of the action-value function,and the action selection strategy are mainly studied,which lays a theoretical foundation for subsequent research in this dissertation.(2)A spectrum sensing algorithm based on DQN(Deep Q Network)is proposed.This algorithm applies the Q-learning algorithm to spectrum sensing.Considering the continuity of the state space,the neural network is used to approximate the action-value function.Simultaneously,existing researches have shown that the user's occupation of the spectrum has a certain continuity,so the primary user's occupation of the spectrum is modeled as a Bernoulli process.To balance exploration and exploitation,this dissertation adopts an improved-greedy strategy,which can fully explore the action space in the initial stage of training,and tends to exploitation more quickly in the middle and late stages of training.In the simulation results,we first compare the performance of the algorithm under different hyperparameters to select the best hyperparameter,then comparing the proposed algorithm with other existing spectrum sensing algorithms,it shows that the proposed algorithm can obtain higher detect probability.(3)Because of imperfect spectrum sensing and multiple secondary users performing spectrum access at the same time.This dissertation proposes a spectrum access algorithm based on multi-agent reinforcement learning,in which multiple agents adopt a centralized control method,and the action space adopts a joint definition method,and the update formula of the action-value function is basically the same as that in a single agent scenario.Considering that the action of agents will interact with each other,we propose an action selection strategy based on game theory.The simulation results show that the proposed algorithm can independently select channels with a higher idle probability.Compared with the existing algorithm,it can increase the overall channel capacity of the secondary user system and cause less interference to the primary user system.