Design And Implementation Of Intelligent Anti-Jamming Frequency Hopping Strategy Based On Reinforcement Learning

With the rapid development of communication technology,wireless communication has become an important part of human society.However,malicious interference can seriously affect the normal function of wireless systems,making it necessary to study communication anti-jamming technology.Traditional anti-jamming technologies are not effective in dealing with efficient and dynamic new types of interference such as clever interference and intelligent interference due to their poor adaptability and flexibility.In recent years,intelligent anti-jamming technology based on reinforcement learning has attracted widespread attention because it can learn interference rules and make optimal anti-jamming decisions.However,there are still some problems in the existing intelligent anti-jamming research,such as:(1)In terms of intelligent anti-jamming frequency hopping,the interference types considered in existing research are relatively single,and there is a dependence on prior knowledge of interference,which limits their usage.(2)Some studies have extended anti-jamming actions to frequency hopping and transmit power adjustment,but transmit power is quantized into a few discrete levels,which can lead to the algorithm not obtaining the optimal solution and affecting communication performance.(3)Existing research mainly stays at the theoretical stage and lacks related research on the design,implementation,and verification of intelligent anti-jamming communication systems.Therefore,this thesis aims to address these three issues as follows:Firstly,a deep Q-learning-based intelligent anti-jamming algorithm is proposed to overcome the dependence on prior knowledge of interference,and to deal with various types of interference without estimating the interference mode and parameters.The algorithm takes the spectrum perception vector and the current communication frequency as the state,learns the interference rules,explores the optimal decision at each state,and is applicable to combat most types of suppression interference.The algorithm is simulated and analyzed in various interference scenarios such as comb,tracking,linear sweep,and dynamic probability,and the results show that the proposed algorithm has strong adaptability,and the anti-jamming effect can approach the theoretical optimum.In addition,this thesis proposes an acceleration method that uses spectrum perception for virtual learning to speed up algorithm convergence.Secondly,for the low-power anti-jamming problem in mobile communication scenarios and the optimal power allocation anti-jamming problem in multi-task transmission scenarios,this thesis studies the frequency-power joint decision intelligent anti-jamming algorithm.To avoid the impact of quantization,a hybrid action space is adopted,which can make discrete frequency and continuous power decisions simultaneously.Corresponding joint decision algorithms are proposed based on the flexible Actor-Critic algorithm for the above two scenarios,and the algorithm is applicable to most types of suppression interference scenarios.The algorithm is simulated and tested in various interference scenarios such as comb,tracking,linear sweep,and frequency-domain dynamic probability,and the results show that the proposed algorithm has good anti-jamming performance and can approach the theoretical optimal decision performance.Thirdly,an intelligent anti-jamming algorithm verification system is designed and implemented.The Q-learning-based anti-dynamic probability interference algorithm is optimized and improved for FPGA implementation,and a flow control logic is designed based on communication frame structure to speed up algorithm convergence.The verification system is constructed to verify the anti-jamming performance of the algorithm in a real frequency hopping communication system under interference scenarios.The convergence time of the algorithm running multiple times was counted,and the convergence performance was tested.The results showed that the algorithm converged in seconds.After convergence,the frequency hopping system can be controlled to avoid interference frequency points caused by dynamic probability interference in the frequency domain,and the anti-interference effect is very good.