An Intrusion Detection System For High-Speed Networks

The rapid development of Internet makes computer network play an important role in people's daily lives. At the same time, caused by the variability, the complicity and the intelligence of the network attack methods, the importance of network security attracts increasing people's concerns while they are utilizing the network. As an active information security protection method, intrusion detection makes up for the disadvantages of the traditional security technology (such as firewall technology) effectively.However, facing the fast developing network size, increasing network flux, the traditional intrusion detection architectures show their limits in many aspects, such as heavy workloads, the slowness of responses, the insufficiency of processing capacity and single point of failure. The paper designs and implementations a distributed intrusion detection system based on Hadoop cluster. In system frame, we used three level frame---as the first level, detection agents detect the security incident; data collectors, process the data from the first level, are the second level; a monitoring center based on Hadoop cluster is the third level.First of all, the paper do some simply researches the definition, importance and methods of intrusion detection, analysis precisely the problems as well as the factors of the intrusion detection system in High-speed networks. The paper also do some introduces three kinds of distributed intrusion detection systems; compare the advantage as well as the disadvantage of these architectures. The paper describes the whole process of the Hadoop cluster technology and explores details of the Hadoop distributed file system (HDFS) and MapReduce programing framework.Then, the paper discusses the design of the overall system architecture. And the paper gives an analysis about the function of components of the system and the key technology of system. After all, the paper elaborates some design and implementation of the components of the system. Finally, the paper offers a Parallel FP-Growth algorithm to deal with massive intrusion records. The algorithm executes in such way that shared a large-scale FP-tree into independent FP-sub-tree. According to mining all sub-trees, we will acquire the global frequent item-sets. The result of experiment shows that the efficiency of algorithm performances linear growth with adding additional computing nodes, when dealing with large-scale data.