Research On Techniques Of Anomaly Detection In High Speed IP Network Based On Traffic Measurement

As the step type increasing of network bandwidth, according to the flow attack events for network security becomes more frequently, and attacking mode becomes more elusive, which make great harm to normal operation of network. How to accomplish the behavior analysis of traffic measurement and anomaly detection in high speed IP network have great importance to improve the network robustness, mastering the network behavior structure and promoting the flourishing development of network architecture. The research works of this thesis is subject to "Three Networks Integration" —the National 863 High Technology R&D Program, which mainly studies on the traffic measurement and anomaly detection of High IP Network and designs a scheme of graded anomaly detection based on traffic measurement. Firstly, the scheme perceives whether there is network attack event by using network safety perception algorithm based on flow numbers estimation. Secondly, through exception-flow selective sampling algorithm based on adaptive sampling to sample traffic which is sensitive to anomaly event. Finally, by using refinement detect model to classify anomaly traffic which based on machine learning algorithm and neural network. This scheme has greatly improved the utilization ratio of system computing resources by the mechanism of processing step and step, and better than the traditional schemes in detection effects. Concretely speaking, the principal results are follows:1. Based on the heavy-tailed property of network traffic a new iterative estimation algorithm is proposed by making a difference between elephant flow and mice to accomplish the coarse-grained perception algorithm of the whole network situation. Through iterative estimation of mice flow which is main factor in flows length distribution, by solving the problems of improving iteration accuracy without speed of iteration renewal, and it has improved about 25 percent to the existing algorithms in complexities.2. The existing sampling algorithms have the problems lied not only in low estimation precision but also in sampling of anomaly traffic. An adaptive flow sampling algorithm based on sampled packets and force sampling threshold S(AFPT) is proposed which is an exception-flow selective sampling algorithm based on adaptive sampling, sampling mice flows whose packets less than S which are sensitive to anomaly traffic, while adaptive adjustment the probability of sampling P(s) based on the sampled packets for flows which have packets more than S. Simulation and experimental results reveal that more than 75 percent of anomaly traffic is sampled, the estimation precision is reduced about 30 percent and the algorithm contributes to improve detection accuracy.3. In order to solve the shortcomings of redundant detection feature dimension and falling into local optimum easily, an anomaly detection model based on modified mutual information-based feature selection algorithm(MMIFS) and Regularization Radical Basis Function(RRBF) neural network is proposed. The model selects the optimal detection feature subset from the raw traffic, training RRBF neural network to establish the model through machine learning algorithm. The simulations reveal that the model has better detection precision and false positive rate. It can escape local optimal and improve learning speed of neural network to some extent.