Research And Analysis Of Network Protocol's Abnormal Behavior

Master the behavior of network protocols is related to the foundation of cyberspace security,it is an important technical prerequisite for the implementation of security protection and offensive and defensive confrontation.With the continuous development of network applications,the behavior of network protocols is becoming more and more complicated,especially many kinds of abnormal behaviors are mixed up with the protocol's normal behaviors.It is difficult to distinguish good from evil,and the traditional protocol reverse engineering and security protections are difficult to deal with abnormal behaviors.Protocol behavior analysis is a reverse analysis technique that analyzing the data characteristics of protocol messages and the instruction sequence characteristics of protocol programs in order to master the protocol's behavior rules.Due to the wide variety of protocol abnormal behaviors,strong concealment and long incubation period,it often does not significantly damage the target host or network,only steal important information secretly in a short period of time,or resume the normal behavior immediately after completing the attack task.At present,the understanding and research of protocol behavior is insufficient,the boundaries between normal behavior and abnormal behavior are difficult to draw.It is difficult to perceive,identify and mining abnormal behaviors such as dormant,hidden and stealth attacks.The abnormal behavior of the protocol poses a greater threat to cyberspace security.Therefore,it is very important and urgent to study the abnormal behavior analysis technology of network protocol.In view of the shortcomings of traditional reverse analysis technology,this paper focuses on the analysis of protocol's abnormal behaviors.Typical abnormal behaviors such as dormant,hidden and stealth attacks are analyzed in this paper.We are committed to researching new protocol's abnormal behavior perception and mining methods,evaluate the protocol's execution security,the main research findings are as follows:1.In view of the problem that the protocol's abnormal behavior is difficult to distinguish,we studied the protocol's different behaviors at the instruction level,proposed the concept of genetic instruction and the problem of instruction mark.All protocol program instructions,regardless of the platform and instruction set,can be divided into three categories of genetic instructions.That is,the function call relevant instructions,which can be marked as F instruction;data processing related instructions,recorded as D instruction;conditional jump related instructions,denoted as C instruction.The different protocol behaviors behave differently on the corresponding instruction sequences,that is,all the protocol behaviors can be expressed as a combination of the three types of genetic instructions.Instruction clustering analysis is performed on the marked behavior instruction sequences,in this way the protocol's abnormal behavior instruction sequences can be exposed,and the protocol's potential abnormal behaviors can be mined automatically.This method can mine the unknown protocol's abnormal behavior with high accuracy.It has a wide range of application scenarios.2.In view of the problem that the incubation period of the protocol's dormant behavior is long and difficult to be exposed,a scheme of combining dynamic taint analysis and instruction clustering analysis is proposed.First,the public behavior instruction sequences of the protocols are captured by dynamic taint analysis,and the general structure of protocol messages are inferred;and then the potential dormant behavior instruction sequences are perceived and mined by the self-designed instruction clustering algorithm,at the same time,the trigger conditions that can trigger the dormant behaviors are mined;according to the mined dormant behavior trigger conditions,combining the general structure of the protocol message that has been inferred,using the self-designed sensitive message generation algorithm to automatically generate new messages that can trigger the protocol's dormant behaviors;finally,the sensitive messages are used to trigger the execution of the dormant behaviors,and the specific content of the dormant behaviors are observed,collected and analyzed.The method can not only distinguish the dormant behaviors quickly,but also can accurately grasp the dormant behavior instruction sequences,and analyze their specific functions,this prepares first-hand information for targeted defenses and countermeasures.3.Based on the above two research work,the prototype system of protocol automatic analysis platform Hidden Disc has been developed independently.Hidden Disc is built on top of the TEMU virtual platform,a user-defined expansion interface is designed,and the programming capabilities can be extended according to the needs of the application scenarios.At present,the three functions of dynamic taint analysis and instruction recording,instruction sequence clustering mining,and sensitive message automatic generation has been implemented.Using this tool,1297 protocol samples are analyzed.Experimental results and comparative studies show that,both in terms of efficiency and accuracy,the analysis platform is ideal for the analysis of unknown behavior such as dormant,hidden and stealth attacks.4.According to the characteristics of the protocol behavior at the instruction level,the concept of behavior distance is proposed.According to the distribution,frequency and quantity characteristics of the 3 kinds of genetic instructions in the protocol behavior instruction sequences,the characteristic distance between unknown behavior and known behavior is calculated,that is,the deviation degree of the unknown behavior relative to the known behavior,thereby determining the abnormal behavior and its characteristics.In view of the problem that the protocol's execution security is difficult to be measured,the protocol's execution security evaluation criteria is proposed.In this paper,we find that the more abnormal behavior that the protocol contains,the greater the distance from abnormal behavior to normal behavior,the potential security threats and hazards are greater,even if one run does not show up,however,the protocol's execution security has been seriously threatened.In this paper,the characteristic distance between abnormal behavior and public behavior is used as an evaluation criterion of protocol's execution security.The closer the abnormal behavior to the public behavior,the more secure the protocol is.On the contrary,the greater the deviation between abnormal behavior and public behavior,the less secure the protocol is.A large number of case studies and comparative studies show that,the protocol's abnormal behavior is significantly different from the public behavior in the distribution of genetic instructions,it is credible and reliable to evaluate the protocol's execution security according to the behavior distance.