Research On Spectrum Decision Algorithm Based On Deep Reinforcement Learning In Cognitive Network

Static spectrum allocation has not fitted the rapid demand.Cognitive radio makes use of spectrum with high utilization by dynamic spectrum access.Spectrum decision involves the whole process of cognitive radio,and deep reinforcement learning can effectively improve the intelligence of spectrum decision.Spectrum decision focuses on different parts depending on different mode of spectrum access,so we study spectrum decision based on different spectrum access mode.First of all,for the problem that traditional power control algorithm needs prior information like the power and its update policy of PU,which is hard to achieve in real environment,we propose an intelligent power control algorithm based on Deep Q Network.The SU can learn effective policy from interaction with environment without much prior information,and target network is set to improve model's robustness.The reward function is appropriate modified to avoid sparse reward problem.Our experiment shows that the SU can intelligently and fast adjust its power to ensure both data transmission.Secondly,for the problem that traditional channel access algorithm cannot guarantee successful access without massive spectrum sense and probe,we propose a fast channel access algorithm based on proximal policy optimization.Actor network updates channel access policy and an old policy network is set to improve sample utilization;Critic network evaluates action policy to guide Actor network's update.Our experiments show that successful access rate of the proposed algorithm can approach that of the algorithm with prior information.Finally,for the problem that Sequential sensing algorithm ignores “poor channels” which have high SNR but the PU often occupies or have high idle probability but low SNR,we propose a fast channel access algorithm based on asynchronous advantage Actor Critic.The local networks interact with environment and upload update information to the global network;the global network updates policy and download to the local networks.And the reward function is appropriate modified to encourage the SU to access channels with higher SNR after meeting the quality of service.