Research On Fuzzying Method Of Industrial Control Protocol Based On Reinforcement Learning

The industrial control system(ICS)is an important part of the national critical information infrastructure(CII).In recent years,with the rapid development of production enterprise informatization and industrial Internet technology,the traditional industrial control system(ICS)independent operation and network isolation mechanism has been broken.The frequent occurrence of security incidents in industrial control systems(ICS)has aroused great attention to the security issues of industrial control systems(ICS).Real-time,reliable and safe network communication is the basic condition for industrial control systems(ICS)to realize remote production scheduling and on-site distributed control functions.Industrial control protocols play a vital role in network communication.The use of effective testing methods to test the security of industrial control protocols is of great significance for the timely detection and repair of potential anomalies or vulnerabilities in the system and avoiding risks in actual operation.At present,the application of traditional fuzzing methods to the discovery of loopholes in industrial control systems is an effective method,but there are some shortcomings:(1)The fuzzing method of industrial control protocol based on known protocol knowledge has high requirements for protocol analysis methods,and has poor real-time performance and low scalability.(2)After generating test cases,many fuzzing methods usually only send test cases to a certain state of the protocol or a single function type code,and cannot perform combined vulnerability mining on test targets.Based on reinforcement learning,this paper proposes a fuzzing method for industrial control protocol to solve the above-mentioned deficiencies.This method can realize online learning of the protocol during the testing process,and achieves the effect of intelligent combination test cases for vulnerability mining.Based on the Q-learning algorithm and the DDPG algorithm,two intelligent combination methods of test cases are designed.One is to select the test case function type code as the action,and the determination of the function type code to the test case is regarded as the environment together with the test object.Train based on Q-learning algorithm to realize intelligent combination of test cases.The other is to consider that a function type code will correspond to different test cases during the test.In order to reduce the complexity and volatility of the environment as much as possible,by comparing the test case generation process,design and implement a test case generation method based on vulnerability characteristics,and use the DDPG algorithm to determine the variable values in the test case generation process,thus achieving intelligent combination of test cases.According to the above methods,simulation environments are setup for experiments and results analysis.The experimental results show that the fuzzing method of industrial control protocol based on reinforcement learning proposed in this paper is superior than the traditional fuzzing method of industrial control protocol: it can realize online learning in the fuzzing process,and intelligent combination test cases carry out combined vulnerability mining on test targets.