Research Of Anomaly Detection Based On Self-similarity Analysis And Feature Classification

The Internet is always affected by the endless emergence attacks, which can be detected byanomaly detection techniques that ensure the security of the Internet. However, anomalydetection method for high speed network is unbalanced between the detection precision and theefficiency, the reason is mainly shown in two aspects:(1)The traditional statistical analysismethod is widely used in high speed network anomaly detection area because of its real timefeature, but whose detection precision is low.(2) The detection method based onmultidimensional feature analysis can’t be used in high speed network because of its highcomputation complexity.The aim of the research is to balance the time and accuracy of anomaly detection methodfor high speed network. Firstly, we proposed a method based on the Fractional FourierTransform (FRFT) analysis and the self-adaption threshold detection for rough detection andnormal traffic filtering. Secondly, a method based on principal component analysis and tabusearch (PCA-TS) and the decision tree classification is proposed for accurate detection. Finally, acascaded anomaly detection scheme based on the methods above is designed. The researches inthe dissertation are mainly as follows:1. A new anomaly detection method based on FRFT analysis and self-adaption thresholddetection is proposed.The traditional self-similarity estimation method has low estimation precision, and the useof fixed threshold leads to low detection precision. To solve these problems, this dissertationproposes a self-similarity estimation method based on FRFT. Based on this method, aself-adaption threshold detection method is proposed, which improves the detection precision byusing a network variation adaptively threshold. Our validation shows that the detection rate ofthe detection method based on FRFT analysis and self-adaption threshold is improved by10%and the positive rate is reduced by5%, and that the method can satisfy the real time detectionrequirement.2. A new anomaly detection method based on PCA-TS and the decision tree classification isproposed.For the “Curse of Dimensionality” problem of the detection method based onmultidimensional features analysis, this dissertation proposes a new feature selection algorithm---PCA-TS based on getting the finding that there are some weak properties and redundantfeatures, which uses PCA for feature set reduction and the tabu search for optimum featuresubset selection. Then a classification algorithm MDDT which has higher classificationefficiency is proposed, and the incremental learning and the label renewal are used in the training stage for higher classification precision. Emulation experiments demonstrate that the overallclassification accuracy of PCA-TS on the Moore dataset is99.38%. We utilize PCA-TS inMDDT and find that the time consumed reduces by27%.3. An anomaly detection system and its realization scheme utilized in the high speednetwork is proposed.To meet the real time and high accuracy requirements of high speed network anomalydetection, a cascade anomaly detection scheme based on the methods above is proposed. TheFRFT analysis and self-adaption threshold detection method is used for filtering most of thenetwork traffic, and the MDDT method is used for accurately classifying and detecting thefiltered lower speed traffic. In the realization scheme, we design three loose coupling detectionmodules for higher flexibility and expandability. The experiments show that the modules canidentify anomalies more than76%and the detection time is less than1minute at the rate of1000Mbps.