An Abnormal Packet Parallel Generation Engine Based On Mapreduce

So far, there are more than5000network protocols that the IETF announced. With the increase in the number of network protocol, the Robustness test is becoming more and more difficult, and it’s also difficult to test and analysis for the single protocol. In order to cover all the test path, the test data set is very large, such as a packet with the header length of40bytes, the test data set up to2^40. So the automated production and test systems are two key factors, both of which can be attributed to the design of test system and the generation of test data set, which are two basic problems for protocol test.Hadoop was introduced as a subproject of the Apache Lucene project Nutch by Apache Software Foundation Inc. in the fall of2005, and Hadoop contains two subprojects, respectively is the Hadoop HDFS and Hadoop Mapreduce. Hadoop HDFS is a distributed file system, and Hadoop Mapreduce is a distributed parellel processing framework. This project based on the open source software Hadoop Mapreduce, and we modified the serial algorithm to parallel algorithm for the generation of abnormal packet, and Hadoop Mapreduce is in charge of its parallel processing, load balancing, fault tolerance. We introduced the fuzzy processing principle in the generation algorithm. The fuzzy processing based on the fields which is parsed by the Wireshark-like module by the transparent design principle of network protocol. The main research include as follow:(1) We analyze the Hadoop Mapreduce parallel processing framework, include the principle and parallel processing procedures;(2) We proposed the field-based fuzzy concept of the test data set and the fuzzy process based on field according to the principle of hierarchical transparent design, and the test data set is more clearer and It’s easy to analyze the result.(3) We designed and realized the parallel abnormal packet generation algorithm by the Mapreduce programming features, and the basis of the parallel algorithm is the original serial algorithm; (4) We study the relation between the parallel algorithm and the serial algorithm, and study the performance impact by the number of PC cluster.