| Penetration testing is a network security testing and evaluation method.By authorizing network security testers to detect the target network in the form of hacker attacks,the potential security risks of the target network can be discovered from the perspective of the attacker.Traditional penetration testing mainly relies on manual methods,which requires a lot of manpower and time costs.Research on automated penetration testing can effectively reduce the cost of network security maintenance and help network security maintainers discover existing security vulnerabilities in a timely manner.Attack path discovery is the key to automated penetration testing.Existing studies are mostly based on attack graphs,attack trees,or partially observable Markov decision models.It is difficult to take into account two aspects: First,the model relies on a large amount of prior knowledge and cannot simulate the penetration testing process from the attacker’s perspective.The second is that the computational complexity of the model is high,and it cannot be well extended to larger-scale network scenarios.In order to simulate the uncertainty of the attacker on the target network environment during the penetration test from the perspective of the attacker,this thesis formalizes the penetration test process as a Markov decision process.This thesis simulates the characteristics of the penetration test process from the design of state space representation,action space and reward function,and uses deep reinforcement learning algorithms to solve the problem of attack path discovery in large-scale network scenarios.In order to solve the problem of attack path discovery in large-scale network scenarios,this thesis proposes an improved Deep Q-Network(DQN)algorithm NDSPI-DQN(decoupling)algorithm,which combines five extensions of DQN reasonably for the sparse reward problem faced by reinforcement learning.The improved version of the algorithm effectively improves the agent’s ability to explore the environment.In order to solve the problem of the huge action space of the agent in large-scale network scenarios,the algorithm decouples the output of the neural network and the action vector of the agent,which effectively reduces the size of the agent action space and the trial-and-error cost in the exploration process.Finally,aiming to solve the problem of attack path discovery in large-scale network scenarios,this thesis builds test scenarios of different scales based on the NASim platform and compares the proposed algorithm with the traditional algorithm.Experiments prove that the algorithm proposed in this thesis has achieved better results in convergence,scalability,and generalization.The experiments verifie the practicability of the research results of this article in network security analysis. |
