Research On Detection Algorithms Of Network Anomaly Traffic Based On Multi-Scales

Multi-scales is statistically one of the most important characteristics. Different scales give prominence to different anomalies which is able to improve detection rate on different scales. Therefore, anomaly traffic detection based on multi-scales is an important research. There are two notoriously important problems preventing it from prevailing abroad: (1) the problems that repeat alarms for the same anomaly and imprecise time in which identified anomalies occur are prone to"alarm storm",making the algorithm paralytic. (2) Because network traffic is usually characterized by its higher-dimensional features, related detectors and classifiers for identifying traffic anomalies are suffering the increased complexity.To solve these problems, this dissertation studies the anomaly detection algorithms based on multi-scales which can satisfy the lower false alarm and online detection. Firstly, we proposed an algorithm based on graphic coverage degree (GCD) to locate the time anomalies occur. And then a multi-resolution low rank (MRLR) model is presented which describes the distribution of anomaly features. Utilizing this model, we can research and design the dimensionality reduction algorithm. The researches in the dissertation are as follows:1. A new algorithms based on GCD to locate the time the anomalies occur is proposedFor"alarm storm"result from repeat alarms for the same anomalies and the imprecise time of identified anomalies, this dissertation proposes a new algorithm --- MS-GCD. MS-GCD combines multi-scales, and separates each scale into the same serial windows. Starting from the biggest scale, we identify sections which are contaminated by anomalies using a global vector which has the same length with the original serils. And then depress scale to the proximal ones and split the observed window into two equal windows. Identify anomaly sectaions on this scale which is also mapped into that global vector. Do that until the smallest scale, and achieves a vectors which contains several disperse anomaly sections. Our validation shows that MS-GCD can provide an accurate anomalies time, whose proportion is more than 70%. Repeat alarms can be reduced to approximately zero.2. A novel model in allusion to anomaly features is proposed --- MRLR"Curse of Dimensionality". Network anomalies are distributed typically in a sparse way. Based on this important finding, this dissertation develops a novel model for detecting traffic anomalies --- MRLR. The proposed MRLR allows us to dynamically filter the"proper"feature sets and then to classify anomalies accurately. We validate MRLR using manually analyzed real traffic anomalies as well as synthetic anomaly injection. Our validation shows that MRLR can accurately filter anomalous flow features, and reduce the dimensions of them to lower than 10%; On the other hand, the complexity of MRLR-classifiers --- FCA are O(n) and no sensitive to feature dimensions; RRF can be generalized to other occasion, because we utilize it in Bayes and SVM, and find that the time consumed reduces by 25% and 30%.3. The schema of an anomaly detection module utilized in the backbone network is proposedWe design and carry out an module of anomaly detection based on multi-scales for network traffic filter systems used in backbone making demands on accurate and real time. Taking agile and extensible requirements into account, we split it into four sub-modules which contact with each other based on database, and avoids coupling with each other. With thousands of simulated attacks, this module can identify anomalies more than 90%. It only consumes less than 100 seconds to process 100,000 traffic traces.