Study On Anti-jamming Communication In Cognitive Radio Networks Based On Reinforcement Learning

Cognitive radio networks are vulnerable to jamming attacks due to the dynamic spectrum access of secondary users and the broadcast nature of wireless channels.By using smart radio devices,a jammer can dynamically change its jamming policy based on opposing security mechanisms;it can even induce the mobile device to enter a specific communication mode and then launch the jamming policy accordingly.In this paper,the interactions between mobile devices and jammers are formulated as an anti-jamming communication game,in which mobile devices can exploit spread spectrum,power control and user mobility to address both jamming and interference.By applying reinforcement learning techniques,we propose a SARSA(?)based anti-jamming scheme,in which a mobile device can achieve an optimal communication policy via trail-and-errors without the need to know the jamming and interference model and the radio channel model.The SARSA(?)based anti-jamming communication scheme can increase the signal-to-interference-plus-noise ratio(SINR)of the signals and the utility of the mobile devices against jamming and interference compared with the benchmark scheme.For example,with 64 channels,the SARSA(?)based scheme improves the SINR of the signals by 13.6%compared with the random based communication scheme.We apply deep Q-network(DQN),a deep reinforcement learning technique,to address the curse of high-dimensionality of reinforcement learning and present a DQN-based anti-jamming scheme.The SINR of the signals and the utility of the mobile devices can be further improved by the DQN-based scheme.For instance,the SINR of the DQN-based is 47.6%higher than that of the SARSA(?)based scheme with 64 channels.Furthermore,we propose a Fast-DQN algorithm,in which a hotbooting deep Q-network scheme is proposed that exploits experiences in similar scenarios to reduce the exploration time at the beginning of the game,and applies macro-action technique to accelerate the learning speed in dynamic situations.The simulation results show that the Fast-DQN based anti-jamming scheme outperforms the other schemes with a higher SINR of the signals and a higher utility due to a faster learning speed.For instance,the Fast-DQN based scheme can further improve the SINR by 12.5%compared with the DQN-based scheme with 64 channels.