Technologies Of Protocol Reverse Analysis On Text-Type Application Layer Protocols

With the networked, intellictualized and fast globalized IT fields developing, network security is becoming a hot research subject.In many important application of network security, such as vulnerability mining, fuzzing test, intrusion detection, and so on, it is a must to first understand the grammer of protocols. By monitoring and analyzing the network input/output of protocol entities and system behavior, protocol reverse engineering can automaticly extract the grammar information and help understand the working mechanism of unknown protocols, which is a very valuable tool for the network application mentioned above. Thus, study on protocol reverse engineering is of great significance.This subject mainly does research on protocol reverse analysis on text type protocols, and it is carried out in the following thoughts. First, the issue sums up the research progess of protocol reverse engineering field in the past ten years, and introduces the conception and application fields of protocol reverse engineering. Then, it systematically analyzes the core technology of existing literatures, that is, the method based on network trace whose core arithmetic is sequence alignment, and the method based on application execution path whose care techonology is dynamic taint analysis. Besides, it concludes the features and shortages of both methods. Based on the above study, this paper puts forward a scheme of protocol reverse analysis on the basis of text mining. The main idea of this scheme is to use related technologies of text mining to analyze numerous protocol messages of the same kind to extract and display protocol syntax.The working results of this paper are as follows. First, it designs complete process of protocol reverse engineering with text mining method, that is, it captures a large number of packets of the same protocol with wireshark tool and extracts the grammer structure of protocol message through pretreatment, segmentation, feature extraction and structure modeling. In words segmentation stage, it devises the segmentation method based on binary tree model. In text feature extraction stage, it presents improved algorithm for TF-IDF. In addition, it adopts the method based on XML for data transmission between different functional modules and description of text structure model.At last, the article shows the realization process of each phase in detail and presents test procedure by taking HTTP and SIP as examples. To verify the solution, it compares the test results with analysis of wireshark tool. The result demonstrates that the schme designed in the subject can conveniently and effectively achieve the goal of protocol reverse, and can truthfully reflects format features of protocol message.