Research On Intelligent Anti-jamming Decision Technology Of Frequency-hopping System

With the deterioration of electromagnetic environment and the intellectualization of interference,the traditional frequency hopping(FH)communication anti-interference technology has been difficult to meet the needs of military communication.There is an urgent need for intelligent anti-interference technology with environmental perception and learning.Agents with learning ability are used in Reinforcement Learning(RL)and Deep Reinforcement Learning(DRL)to interacted with the environment and learn it.As a result,they can adapt to the highly dynamic complex environment and obtain the optimal decision according to their states.They have been widely used in wireless communication.Therefore,the intelligent anti-interference decision-making technology of FH communication system based on RL and DRL is studied this paper.Firstly,the intelligent anti-jamming decision-making problem of FH communication system in complex electromagnetic environment is studied.The FH user is regarded as an agent,and the decision-making problem is modeled based on Q-Learning(QL).Then,the Heuristically Accelerated Q-Learning with Optimal Solution Guidance Mechanism(HAQL-OSGM)algorithm is proposed.Aiming at the slow convergence speed and the difficulty of balancing exploration and utilization of RL,the heuristic function is designed based on the optimal solution guidance mechanism.Then,the new heuristic function and the sigmoid function are introduced into the action selection strategy of QL.HAQL-OSGM algorithm can balance exploration and utilization better and has faster convergence speed.The simulation results show that,compared with the traditional methods,the intelligent anti-interference decision algorithm of FH communication system based on HAQL-OSGM has obvious advantages in the convergence speed,the convergence performance and the ability to break away from the local optimal solution.And it is adaptive to changeable interference environment.Then,the intelligent decision-making of bivariate FH pattern is studied.Aiming at the problems which traditional FH pattern design methods faced such as too many preconditions,low parameter flexibility and weak ability of counteracting complex interference,the Deep Q-Network with Priority Experience Replay Based on Pareto Samples(PPER-DQN)algorithm is proposed for intelligent decision-making of bivariate FH patterns.Deep Q-Network(DQN)is used in this algorithm to increase the flexibility of FH pattern parameter decision-making.And the Priority Experience Replay(PER)technology and Pareto optimal solution theory are used to improve the learning efficiency of the algorithm.The simulation results show that in the intelligent decision-making of bivariate FH pattern,PPER-DQN algorithm can screen samples more efficiently,improve the convergence speed of the algorithm effectively.The generated bivariate FH pattern has stronger and more robust antiinterference performance than conventional FH pattern.Finally,the intelligent anti-interference decision-making of FH asynchronous networking communication is studied.In order to solve the problems of low efficiency of spectrum resource allocation and poor ability of anti-complex interference in traditional FH network communication,each subnet is regarded as an agent,and the Multi-agent Fuzzy Deep Reinforcement Learning based on Centralized Training and Decentralized Execution(MFDRL-CTDE)algorithm is proposed.Centralized training and decentralized execution framework are introduced to help the multi-agent system to adapt to the unstable environment and improve the efficiency of agent mutual learning.A fusion weight allocation mechanism based on fuzzy inference system is proposed to solve the problems of difference and cooperation between agents.The Priority Experience Replay technology and dueling network structure are also introduced to improve the learning efficiency of the algorithm.Simulation results show that MFDRL-CTDE algorithm has faster convergence speed,higher and more stable convergence performance.And it has better adaptability to changeable and complex electromagnetic environment.