Research On Network Anomaly Detection Algorithms

How to detect anomalies timely and accurately is of great significance to network security.Compared with traditional misuse detection which is based on feature matching,anomaly detection has the advantages of small computational load,the ability to detect zero-day attacks and the feature of being not influenced by encrypted traffic,thus it becomes a hot academic topic in recent years.So far,current anomaly detection methods commonly have the following drawbacks:1)the detection rate is low and the false alarm rate is high;2)model establishment and selection is not easy;3)the effectiveness of the detection model can be affected by network anomalies and burst traffic;4)it is difficult to meet the requirements of real-time detection on backbone link traffic.Based on the analysis of network traffic,this thesis proposes several efficient anomaly detection algorithms,conducts a deep research on the model selection problem,and makes a preliminary exploration into the combination of large data technology and anomaly detection.The main contributions of this thesis are as follows:1.This thesis extracts two linearly correlated metrics-IGTE(Inter-Group Traffic Entropy)and IGFE(Inter-Group Flow Entropy)-from the network traffic,and propose an anomaly detection algorithm based on linear regression equation.In order to include more traffic features and analyze the network traffic from multiple perspectives,this thesis later extends the linear regression based detection algorithm to the nonlinear case,and proposes an algorithm based on the evidence function which can determine the optimal model order automatically.2.In this thesis,we observe that data pollution can lead to performance degradation of the linear regression based detection algorithm.We propose a M-estimate based algorithm which is robust against data pollution.3.We propose two detection algorithms based on Least Mean Square Adaptive Filter.These two algorithms are able to output detection results based on partial data and to adaptively adjust the weights according to the transient states of network,thus are suitable for real-time detection.4.In order to improve the robustness and reliability of anomaly detection for different network environments and different anomalies,we present an integration algorithm which combines the results of multiple detection algorithms,and implement the integration algorithm based on the computation system Apache Storm in real time,thus verify the feasibility of performing real-time anomaly detection by using big data technology.