Reseach On Identification Technology Of The Application Layer Based On Hadoop

Along with the rapid development of the Internet, the application layer protocol of all kinds emerge constantly, which makes the network more complicated, more diverse, and more difficult to manage. All kinds of attacks emerge in endless, malicious attacks have caused serious damages to network services and information security. The identification technology of the application layer protocol can solve the problem of real-time identification and the extracted features of the network traffic.Taking account of the security and flexibility, lots of new application layer protocols prefer dynamic port to fixed port by transmitting data. In addition, many agreements do not have unified standards or criteria, so we can’t find simple and unified classification rule. The classification based on the port number is not applicable to dynamic port, and the classified method based on load will involve privacy of user at the expense of lots of time. The extraction of the regular expression is mainly based on specification document of manual analysis of the application layer protocol. In today’s data explosion, artificial analysis of protocol for feature extraction has become increasingly difficult.In view of the current problems, this article puts forward the recognition system of the application layer protocol based on Hadoop. Using the Hadoop, which can process huge amount, can extract and identify the characteristic of application layer packet by identifying application layer packet and extracting the characteristics of the data packets. The main contents of this research are as follows:Firstly, this article makes a research on the existing application layer protocol identification technology, with the architecture and work mechanism of Hadoop and HBase.Secondly, this article makes a research on Apriori. The thesis puts forward an extraction method of application layer protocol creature string based on Hadoop, namely, MapReduceApriori. The method can solve the problem of extracting characteristics from the protocols that unpublished documents. This method can be used to solve difficulty extracted from non-public specification document in the application layer protocol, and the difficult problems of new protocols.Finally, the research designs and realizes the application layer protocol identification system based on Hadoop. The experiment shows that, the system can identify the application layer protocol efficiently and accurately. Moreover, it can extract the character string of the unidentified protocol accurately.