Research On Intelligent Anti-jamming Decision Technology Of Frequency-Hopping Communication

Frequency hopping communication has good anti-jamming ability and anti-reconnaissance ability,and has been widely used in civil and military fields.With the increasing complexity of the electromagnetic environment and the development of artificial intelligence,intelligent frequency hopping communication has attracted people’s attention.As a model-free and unsupervised learning method,reinforcement learning can adapt to parameter decision-making tasks in dynamically changing electromagnetic environments of complex interference,and has been widely used in the field of wireless communication.Therefore,the intelligent anti-interference decision-making technology of frequency hopping communication based on reinforcement learning is mainly studied.Firstly,the intelligent decision-making of frequency hopping parameters such as frequency hopping rate,channel bandwidth,and frequency hopping sequence is studied.Based on the background of blocking interference and sweep interference,the corresponding decision model,state-action space and reward function are designed,and an intelligent anti-interference decisionmaking algorithm based on improved SARSA learning is proposed.Aiming at the problem of the low exploration rate of the environment and the low efficiency of value function update of traditional SARSA learning,the action selection strategy based on upper confidence bound(UCB)and prioritized sweeping are introduced into SARSA learning,which improves the exploration rate and value function update efficiency of the algorithm.The simulation results show that the algorithm better balances exploration and utilization,improves the sample utilization rate and control learning ability,has stronger adaptability and stability to different interference environments,and can effectively improve the energy efficiency of the frequency hopping system.Then,the intelligent decision-making of the bivariate frequency hopping pattern of variable channel bandwidth and variable frequency hopping rate in the bivariate frequency hopping system is studied.Aiming at the problem of weak anti-jamming ability and anti-reconnaissance ability of traditional frequency-hopping pattern,the PPO based on Weighted Importance Sampling and Eligibility Trace(ET-PPO)algorithm is proposed for intelligent decision-making of bivariate frequency-hopping pattern.Aiming at the problem of the high variance of the sample update mode of the PPO actor network and the slow convergence speed of the PPO critic network,the weighted importance sampling and eligibility trace methods are introduced into the PPO,which reduces the sample update variance and avoids the algorithm falling into the local optimal solution.Aiming at the problem that the PPO action selection strategy is not suitable for a limited range of action space,the action selection strategy of Beta distribution is introduced to improve the exploration ability of the algorithm.The simulation results show that ET-PPO improves the learning efficiency and convergence performance of the algorithm,has stronger adaptability and stability to different interference environments,and has better performance of the bivariate frequency hopping pattern.Finally,the intelligent decision-making of the joint frequency hopping pattern of each subnet in the asynchronous dynamic orthogonal networking communication is studied.Aiming at the problem of low networking flexibility and frequency band utilization of traditional frequency hopping networking mode,the QMIX Based on Dataset Aggregation and Options Architecture(DO-QMIX)algorithm is proposed for intelligent decision-making of multi-subnet joint frequency hopping pattern.Aiming at the problem that QMIX is easy to fall into local optimal solution and has low sample utilization,the dataset aggregation technology and options architecture are introduced into QMIX,which improves the learning speed during the early stage of the algorithm and the convergence performance during the later stage of the algorithm.The simulation results show that the DO-QMIX algorithm has good scalability to the number of agents,has stronger adaptability and stability to different interference environments,and has better performance of the joint frequency hopping pattern.