Based On The Improved Q Learning Cognitive Wireless Network Research Dynamic Spectrum Access Algorithm

Cognitive radio network (CRN) is an intelligent wireless communication networkbased on cognitive radio (CR) technology to improve spectrum utilization. Dynamicspectrum access (DSA) is one of the key problems in CRN, which focus on how the CRusers use the authorized frequency band reasonably and efficiently in the dynamicenvironment. Considering that the Q learning algorithm of reinforcement learning hasthe ability to learn autonomously, we apply the multi-agent Q learning (MAQL) methodto DSA of CRN, and propose several DSA algorithms to be used in different models.Firstly, we introduce the theory and model of DSA in CRN, and discuss the Qlearning theory. Then we map the cooperative MAQL to the DSA algorithm of CRN,and propose the corresponding algorithm framework.Secondly, based on the ε-greedy policy of MAQL, we propose two DSAalgorithms in the sharing and exclusive mechanisms respectively. The one iscooperative ε-Greedy_MAQL DSA algorithm based on CR user’s sharing mechanisms,and the other is cooperative ε-Greedy_MAQL DSA algorithm based on CR user’sexclusive mechanisms. Simulation results show that the two algorithms achieve goodperformances in terms of throughput and fairness.Thirdly, to balance exploration with exploitation in the leraning process, we extendthe single agent Q learning based Metropolis criterion of Simulated Annealing to themulti-agent Q learning. We propose an improved algorithm: SA_MAQL. Then wepropose two DSA algorithms: the cooperative SA_MAQL DSA algorithm based on CRuser’s sharing mechanisms and the cooperative SA_MAQL DSA algorithm based onCR user’s exclusive mechanisms. Finally, we compare cooperative SA_MAQL DSAalgorithm with cooperative ε-Greedy_MAQL DSA algorithm in several scenarios. Theresults show that the proposed algorithms have a better performance in terms ofthroughput, conflict probability, fairness and convergence than cooperativeε-Greedy_MAQL DSAalgorithm.