Research On Key Technologies Of Protocol Reverse Engineering Based On Model Learning

In recent years,the widespread use of applications has brought many conveniences to human life,but at the same time,it has also given an opportunity to cyber attackers who spread malware.Malware has become one of the top priorities in the field of cybersecu-rity because of its ability to steal sensitive personal information,paralyze systems,and be used in large-scale cyber-attacks,posing a threat to the cybersecurity of individuals,en-terprises,and even countries.According to existing research results,malware often uses private protocols to communicate and transmit data.Due to the lack of specification doc-uments for private protocols,obtaining their semantic information and behavioral models becomes difficult,which poses a significant challenge to the research of private protocols and defense against malware.In order to solve the problems of lack of private protocol research methods and dif-ficulties in malware detection in the current network environment,based on the existing research foundation and results,we deeply study the model learning technique of the black box system,the symbolic execution technique of the binary program,and the complex sequence alignment technique,and propose the Protocol Reverse Engineering Based on Model Learning(Pro Learn)framework and a sequence alignment based state machine comparison method.Specifically,the research work in this thesis includes the following two parts.1.At the framework design level,this thesis proposes Pro Learn,a protocol reverse framework based on model learning,to address the problems of difficult abstraction of black-box systems in realistic scenarios,lack of a priori knowledge and insufficient in-verse analysis capability of existing methods.The framework is divided into two parts:the“Teacher”implementation method based on symbolic execution and the protocol re-verse method based on the L_a~*algorithm:the Teacher implementation method based on symbolic execution completes the abstraction of the black-box system and the revelation of the protocol alphabet information,which ensures the stability and rationality of the framework;the L_a~*algorithm achieves the protocol reverse engineering of the unknown binary program without accessing the protocol communication entities and network traf-fic,completing the keyword recognition and state machine inference of the communica-tion protocol of the unknown binary program,and ensuring the accuracy and completeness of the protocol reverse results;2.At the experimental design level,in response to the lack of current state machine comparison methods and the single and imprecise evaluation index of state machine infer-ence results,this thesis proposes a multi-type state machine comparison method based on sequence alignment,which implements a comparison scheme between different types of state machines and designs a more precise index to evaluate the similarity between state machines.ensures the accuracy of experimental analysis.Based on the above research work,a comparative experimental analysis is carried out in this thesis,and the experimental results show that the protocol reverse performance of the Pro Learn framework is better than the existing methods in both cases.In terms of protocol keywords recognition,the accuracy rate reaches more than 90%;in terms of pro-tocol state machine inference,the number of state machine states inferred by Pro Learn is closer to the standard model,and the sample acceptance rate indicator reaches more than90%;the similarity between Pro Learn inference results and the standard model of proto-cols reaches more than 90%for simple text protocols,and more than 80%for complex binary protocols.