Research On Dynamic Spectrum Access Algorithm Based On Q-learning

With the rapid development of wireless communication technology,the number of terminals and communication services has increased dramatically.Due to the popularity of ubiquitous communication,the contradiction between spectrum resource demand and scarcity is increasingly serious.Cognitive radio has brought solutions to the dramatic increase in spectrum efficiency.One of the key technologies of cognitive radios,dynamic spectrum access can enable secondary users to intelligently perceive and dynamically access the optimal frequency band based on the real-time status of the channel.It is an important measure to solve the scarcity of current spectrum resources and the low utilization rate.For the problem of dynamic spectrum access of secondary user,a dynamic spectrum access algorithm based on Q-learning is designed.The key of the algorithm is to take advantage of adaptive ability to guide secondary user to continuously interact with the environment and select the channel with the highest rate of return quickly for data transmission.Dynamic spectrum access is a dynamic time-varying optimization problem.Q-learning algorithm,as a model-free online reinforcement learning algorithm,does not need to model the external environment,so it is very suitable for application in dynamic spectrum access systems to guide the secondary user select the channel.First of all,for a complex spectrum environment,by setting various parameters in the Q-learning algorithm and the action strategy,the secondary user can accumulate experience in the process of continuous interaction with the environment,and select the optimal channel access.It has a significant effect on improving the throughput of secondary user and reducing the probability of collision with the primary user.Then,in the 2.4G-band WiFi channel scenario,a WiFi analyzer is used to collect WiFi channel state data in three different user-intensive environments and test the channel quality,consistent with the optimal channel selected by the Q-learning algorithm.Finally,we have programed and simulated on the experimental platform based on the NI USRP-2932 in Lab VIEW.In the WiFi channel scenario,information is sent and received for testing.In the experimental process,secondary users can adaptively select the optimal channel according to the current channel environment.Through comparison and analysis,the effectiveness of the algorithm is proved again.Through the theoretical derivation,experimental testing and virtual simulation,the three different methods have proved that the Q-learning algorithm can be used to select the current optimal channel access and communication according to the real-time channel environment when applied to the dynamic spectrum access.In the process of communication,the monitoring of the channel is maintained,when the primary user has a communication request,the channel is released in time to avoid conflict with the primary user.At the same time,it immediately guides the secondary user to re-select the new optimal channel to complete the communication task and achieve seamless connection.It has a good effect on improving spectrum efficiency,improving throughput,and reducing the probability of communication errors.In summary,the simulation and experimental results show that the Q-learning algorithm can be one of the solutions for dynamic spectrum access in ubiquitous wireless systems.