The Study Of The Dynamic Spectrumallocation Algorithm Based On Reinforcement Learning

Cognitive engine is the intelligent core of cognitive radio which execute the process of simulation, learning and optimization which is used in reconstruction in the communication system.Cognitive engine must have the ability of learning which is the key component distinguishing cognitive radio from traditional radio. In this paper, reinforcement learning is proposed to solve the problem of dynamic spectrum access by allowing the cognitive users to have some intelligence and make the cognitive engine to have the ability of reasoning.In this paper, the fundamental principle of reinforcement learning is introduced.Several familiar reinforcement learning algorithms are described, and the theory of multi-agent reinforcement learning is discussed in dedail. All these lay a solid theoretical basis for further studies.This paper takes SNR of each channel into account in the reward function r and put forward a improved DAQL algorithm which is proposed to solve the problem of dynamic spectrum access.This improved DAQL algorithm allows cognitive users having some intelligence. Simulation results show that this improved algorithm can reduce probability of conflict among users. Furthermore when considering the SNR this algorithm can enlarge the system's average capacity.This paper studies the problem of multi-user dynamic spectrum access mostly. This paper puts forward an algorithm of dynamic spectrum access of multi-user based on independent learning. In this algorithm, each cognitive user is an agent based on independent learning. It only protects their own Q-value table of state-action without knowing the actions of other cognitive users in joint operations. Each cognitive user takes their own independent iterative process. Meanwhile this paper takes SNR of each channel into account in the reward function r.Simulation results show that this algorithm can reduce probability of conflict among uses. Furthermore when considering the SNR this algorithm can enlarge the system's average capacity. In order to accelerate the learning rate and have a better convergence rate, an algorithm based on cooperation learning is proposed with blackboard model, fusion algorithm and reinforcement learning algorithm unified. In the algorithm, the blackboard is a memory region that may realize information sharing.The fusion algorithm is used to fusion to the shared information, and reinforcement learning algorithm is used to select action with the fused result. Simulation results show that this algorithm have more faster learning rate and better convergence rate than the algorithm of dynamic spectrum access of multi-user based on independent learning. This algorithm can satisfy the demand of cognitive radio.