Research On Fuzzing Optimization Technology Based On Machine Learning And Particle Swarm Algorithm

With the rapid development of Internet technology and its application fields,the problem of network security has gradually attracted the attention of the state and society.As an automated vulnerability mining technology,fuzz testing technology has gradually attracted everyone ’s attention in recent years.Most of the existing AFL improvement methods for fuzzing framework only consider one stage.For example,the machine learning application test case filtering stage can select more effective test cases to improve the efficiency of fuzzing.However,it does not fundamentally change the mutation strategy of test cases,and cannot significantly improve the AFL vulnerability detection ability.For the AFL mutation strategy,there are many neural network-based path information-oriented test case mutation methods such as NEUZZ,but they also have invalid test cases.This thesis proposes an optimization algorithm Trans-PSO-AFL based on machine learning and particle swarm optimization algorithm for fuzzy test vulnerability mining technology.The algorithm combines the optimization of the test case filtering algorithm with the optimization of the mutation strategy,which not only changes the test case mutation method,but also filters the invalid test cases generated by the mutation,so it can optimize the above defects to a certain extent.Trans-PSO-AFL optimization algorithm is improved from two aspects of the fuzzing framework AFL.One is the test case filtering stage optimization algorithm Trans AFL.The second is the mutation operator selection stage optimization algorithm PSO-AFL.Specifically speaking,the Trans AFL algorithm selects excellent test case execution by using the excellent seed feature model obtained by Transformer deep learning network training,and discards invalid test cases,so as to reduce the time and resources spent on executing invalid test cases and improve the efficiency of fuzzing.The PSO-AFL algorithm uses the optimal mutation operator obtained by the iteration of the improved particle swarm optimization algorithm to select the probability distribution to guide the mutation of the test case,so as to optimize the mutation strategy of the test case.Through comparative analysis of experimental data,under the application data set PData Set and the data set LAVA-M,Trans AFL,PSO-AFL and Trans-PSO-AFL optimization algorithms are superior to AFL in performance,while the combination optimization algorithm Trans-PSO-AFL has the best performance.In the LAVA-M dataset,the number of vulnerabilities found by Trans-PSO-AFL accounts for 75.6% of the total number of known vulnerabilities in the LAVA-M dataset,8.8% higher than NEUZZ.In the PData Set dataset,the number of vulnerabilities found by Trans-PSO-AFL is 35.3% higher than Trans AFL,59.4% higher than PSO-AFL,and the code coverage is 50.1% higher than the AFL average.