Research Of Network Security Dynamic Defense Based On Q-learning Mechanism

In large-scale network attacks,there are mainly two key issues in order to effectively implement dynamic deployment of attacks and defenses in the network environment.Firstly,network status monitoring,secondly,selection of defense opportunities.Attack graph technology is an association analysis technology established for the network vulnerability.It can perform security analysis and dynamic defense on the network environment.In this study,through the analysis of the Q-learning mechanism,the agent uses the inference engine to simulate the behavior of the attacker.The agent uses the Q-learning mechanism to explore the attack graph environment and generates an attack graph during the exploration process.This paper establishes a dynamic defense model based on the safety distance.A software was developed using the algorithm and model in the paper.Specifically,the main contributions of this article are as follows:(1)An algorithm for generating attack graphs based on the Q-learning mechanism is proposed.The algorithm mainly solves the problems of low efficiency and generation of small-scale attack graphs for traditional attack graph generation algorithms.According to the characteristics of the Q-learning mechanism,the attacker is simulated as an agent to perform continuous exploiting attacks on the network.Simulate the attack path in the attack graph as an agent learning scenario.Since the scenes of the agents are stateless,the generation of attack graphs can be performed in a distributed manner.(2)In the network security defense,this article takes into account that the repair costs of some services and functions are relatively large.Network defense is not blindly fixing bugs and resetting firewalls.This paper combines the attacker's attack action and Q-learning mechanism to propose a dynamic defense strategy for different security risks.(3)The dynamic defense software of the Q-learning mechanism was designed.The software was developed in a way that the front-end and back-end were completely separated.The software also makes use of several common design patterns,such as the singleton pattern and the builder pattern application on the back end of the Q-learning engine.Through the front-end framework React,a component-based single-page application is developed.This front-end mainly completes the generation of attack graphs for a specified network and the real-time visualization of the dynamic defense model of the security distance.