Research On Detection And Control Of Anomaly Traffic In Network Core Nodes

The rapid development of information impulses the society's progress and network has become one of most important infrastructures. The network security and its operation have a great effect on the social normal activity. With the development of network and its applications, malicious behaviors, such as network intrusion, services attack, information stealing, virus propagation and etc., become ubiquitous, numerous and unquenchable for various profit motivation. Among them, the attack consuming resources, such as DDoS, poses a serious threat because the attack can be easily realized but difficult to defend against. The tradition Intrusion Detection System is usual deployed at the user side to protect users, but can not eliminate the malicious traffic in core nodes. Without the ability to detect and defend against the malicious traffic, most core nodes only let the traffic pass. The development of network can hardly keep up with the steps of the resources misusing with the addition of worm and P2P traffic, therefore it is significant to deploy the defense mechanism in core nodes which can be the stations to trace back the attackers in the future.Core nodes can only occupy small parts of computation and memory resources for the additional detection and control functions because their main tasks are routing and switching packets. In the face of the huge traffic, the rigid time requirement and the attack diversity bring challenges to detection. How to deal the data streams and keep the validity and veracity become the key objects of this thesis. The main contributions of this thesis include the following aspects:1. A framework for anomaly traffic detection and control in core nodes is first proposed in this thesis and the framework is composed of three levels. The low level just detects flow with huge multiplicity in traffic according to one pass and data synopsis in data streams model. The low level makes a simple check to achieve the online dealing. The middle level uses aggregate technology in pattern recognition and releases the dependence to the attack prior knowledge. The middle level does not work in a strict real time style, but it can improve the efficiency and validity of results while delaying the reactive time. The top level executes the multilevel aggregation with protocol's model and partitions the malicious degree of each cluster. The resources are allocated and the result is feed back according to the malicious degree. With the combining three parts, malicious traffic can be detected immediately and be filtered efficiently.2. A burst traffic detection algorithm based on skew degree on sliding window is proposed. It aims at large multiplicity flows and designs a cascaded data summary structure base on hash counter matrix. The summary belongs to sliding window model and updates while each data arrived. Each bucket in exponential histograms computes itself skew degree to indicate the degree having large multiplicity flows. The different level bucket in exponential histograms has different size, therefore the data time decay is considered and the skew degree is a well indication that can infer the burst in time.3. The two factors of the tradition counter bloom filters in data compression are improved to adapt the detection of the large multiplicity flows. Firstly, the fixed counter is changed to the logical counter that can intrude the other counters in order to store more large value while it is overflow and the change causes the false negative little worse. Secondly, another improvement is that the null counters in bloom filters are computed to estimate the optimal number of hash functions instead of fixed number, so the computational complexity is reduced.4. A two-stage clustering algorithm based on attack analysis visualization is put forward to deal with the difficulty of signature matching brought by variability of resource-consuming attacks. Since the special attack packets have high similarity in distance and distribution, the first stage of the algorithm gets micro clusters according to the distance and the second stage of the algorithm executes density aggregation according to the distribution similarity. The algorithm can identify arbitrary shapes and distinguish different density areas, therefore it can work with various attacks.5. To control the allocation of resources, this thesis designs multilevel aggregate rate limit mechanism that partitions traffic with protocol patterns. The more anormaly the flow is, the less resource it gets. The special filter can be supported by the nice rules that mined from the aggregate results in the middle level. As well as the feedback of control result, multilevel aggregate guarantees the isolation of different protocol applications.