The Research And Implementation On Load Balancing Of Parallel Network Intrusion Detection

With the rapid development of computer communication technique, the network becomes the most important infrastructure of information. Until the end of June, 2008, the number of Chinese Internet users has reached 253 million, and becomes the first one in amount in world. However, because of the bug of network protocol, OS, the wrong configuration and improper usage of the Internet, the security of Internet becomes more and more a serious social problem. Meanwhile, the link speed increases exponentially from 10Gbps to 40Gbps, and soon 100Gbps which is on research. For the whole network security situational awareness, it's important to dispose an efficient and real-time intrusion detection system (IDS) on the high speed backbone link. But the intrusion detection system (IDS) based on the universal computer system can only handle 100Mbps because of its software bottleneck. Thus parallel intrusion detection system based on existing IDS sensors becomes a hot topic of research.This thesis summarizes IDS, analysis the existing parallel IDS architecture on the first. Then grounding on the packets of an attack can reach the target through different paths, we proposed the architecture of multi-path converge hierarchy switch. How to design a load balancing scheme suitable for parallel network intrusion detection is a key point for parallel network intrusion detection. The parallel network intrusion detection has particular demands to load balancing: guarantee of the context of one attack, the minimization of discard packet rate, load balancing, the security of the load balancer and the minimization of packet reorder rate. This thesis analyzes the existing load balancing scheme, especially these suitable for parallel network intrusion detection.This thesis discuss the load balancing pretreatment distilling the tuple-5 information form POS packet, including parsing the POS packet and deal with the fragmented IP packet. Though investigating the fragment characteristics of real Internet traffic, we find that the number of IP packets that some of its fragmented packets reach the load balancer and others not yet is few. Based on this, we propose a collide then try fragmented IP packets dealing scheme which hash srcIP, dstIP, ID, TYPE to the address of a small fragmented-RAM, when hash collide, the scheme attempt another address which is prime distance from the earlier one. Validated by simulation, the scheme consumes very small memory, comparing with the perfect scheme that has no collision, its performance recede very little.By investigating the characteristics of Internet traffic, we discover that flows with moderate traffic are few but they have large proportion of the total bytes. So we propose a novel load balancing scheme named HAIF (Hashing Adapted by Intensive Flow), which only adjusts the IFs (Intensive Flows) when unbalanced. It only adjusts the few IFs and disrupts flows at very low rate. At the same time, it uses the traffic threshold to identify the IFs, that is, a flow is added to the IFT (Intensive Flows Table) and adjusted by the scheme when it reaches the traffic threshold, so it can adapt the burst characteristic of Internet traffic, it discards packets at very low rate. On the whole the scheme satisfies the demands of parallel network intrusion detection. Validated by simulation, Comparing with the two static hash schemes, HAIF exhibits its advantage in reducing packets discarding and balancing the load, it improves these two aspects by about 100 times. Comparing with SHI scheme presented by Weiguang Shi, our scheme has better performances, but it consumes more memory within the tolerable range.Finally, we implement a Load Balancer based on FPGA using the HAIF scheme. The Load Balancer can evenly dispatch the backbone traffic of 40Gbps to four 10Gbps traffics, which are detected by 10Gbps parallel network intrusion detection systems that are developed earlier. The load balancer is already used in our project.