Research On Reinforcement Learning Technology In Cognitive Radio

Cognitive radio is a technology to improve spectrum utilization,and dynamic resource allocation is one of its key technologies.The reinforcement learning algorithm can get the optimal decision matching with the environment through the dynamic interaction with the environment,and has been widely used in the cognitive radio field.This paper mainly studies the dynamic resource allocation algorithm based on single-agent Q learning and multi-agent Q learning.Firstly,the single-agent Q learning algorithms applied to centralized cognitive wireless networks are studied.Aiming at the problem of low resource allocation efficiency of traditional single-agent Q learning algorithm in centralized network,a pheromone stringency based heuristically accelerated Q learning(PS-HAQL)algorithm is propose,by introducing information intensity in the heuristic function,highlighting the actions of good performance,reducing the unnecessary exploration of the Agent,and improving the convergence speed of the algorithm.At the same time,the improved heuristically accelerated Q learning(IHAQL)algorithm based on the guidance function is proposed.The simulation results show that the PS-HAQL algorithm outperforms the traditional Q learning algorithm,and the IHAQL algorithm outperforms the PS-HAQL algorithm.Then,the multi-agent Q learning algorithms applied to distributed cognitive wireless networks are studied.Aiming at the influence of independent learning between different Agents on convergence speed and system performance in traditional multi-agent Q learning,an incomplete cooperative Q learning algorithm(ICQL)and a completely cooperative Q learning algorithm(CCQL)are proposed.Each Agent learns from each other through the sharing and integration of Q values in different ways,and then the resources allocation of distributed network is speeded up.The simulation results show that the performance of the cooperative Q learning algorithm is better than the traditional independent Q learning algorithm,and the performance of the completely cooperative Q learning algorithm is better than the incomplete cooperative Q learning algorithm.Finally,four combined algorithms are obtained by combining the above proposed four improved Q learning algorithms with case-based reasoning techniques to achieve resource allocation.Since the Q value of the traditional Q learning is initialized to 0,the good action and the bad action are treated equally,which affects the optimization speed and the optimal solution's performance of the Q learning.In contrast,this paper selects the most similar case to initialize the current problem,and uses the above four improved Q learning algorithms to perform iterative optimization for resource allocation in centralized and distributed networks.The simulation results show that the performance of the combined algorithm is better than that of PS-HAQL?IHAQL?ICQL and CCQL,respectively.