Research And Implementation Of Anomaly Detection System Of Network Traffic Based On Data Mining

In recent years,the Internet has become the most important infrastructure that is omnipresent in our daily life and work.However,modern society relies heavily on the Internet,leading to a steady increase of network attacks,both in terms of quantity as well as quality.With the wide applications of data mining technology,new ideas and methods have emerged in detecting network anomaly traffic.The features of network anomaly traffic must be more or less different from the features of normal traffic,so the abnormal state of the network traffic can be analyzed by the difference of traffic features,and these differences can be expressed using the data mining rules.Therefore,network traffic anomaly detection based on data mining is an important research topic in the field of intrusion detection.At present,the network traffic anomaly detection system based on data mining mainly has the following problems(1)The traffic anomaly detection systems usually treat the overall traffic as processing object,but because of the high traffic features dimension,it leads to the high computational complexity of the system;(2)P2P traffic and anomaly traffic have high similarities in behavior,the existence of P2 P traffic will significantly reduce the detection rate of anomaly detection system.However,most of the traffic anomaly detection systems did not consider this effect.Aiming at these problems,this dissertation has researched the following main contents:1.Aiming at the problem that the dimension of network traffic features is high and the original traffic features need to be reduced in data mining,a hybrid feature selection algorithm based on approximate Markov blanket is proposed.By decoupling relevance and redundancy analysis,the whole process of selecting relevant feature subset and removing redundancy is studied in detail.Finally,the performance of the proposed algorithm is verified on the NSL-KDD datsaset,compared with other feature selection algorithms,the proposed algorithm can not only obtains high classification accuracy,but also the least number of selected features.2.Considering the existence of P2 P traffic can significantly reduce the detection rate of anomaly detection system,the P2 P traffic identification method is studied.A rapid P2P traffic identification method based on decision tree is proposed.According to the characteristics distribution of packets payload length in P2 P applications,seven features which can be quickly acquired in UDP flow are selected.Combined with the UDP port heuristic feature,a P2 P traffic identification model based on C4.5 is constructed.Finally,the simulation results on real P2 P traffic show that the model can accurately and quickly complete the classification.3.Considering single feature analysis is not enough to accurately describe the anomaly traffic,network anomalies are represented by multidimensional features.A network traffic anomaly detection and classification method based on multidimensional features analysis is proposed.We study the behavior features of anomaly traffic in depth,and analyse the anomaly traffic from six different feature dimensions.And the simple combination of features will lead to low detection rate and high false alarm rate,the ensemble learning Ada Boost algorithm is used to further improve the performance of decision tree classifier.The simulation verification on real traffic shows that the constructed model has good anomaly detection performance.4.Aiming at the research and development requirements of anomaly detection system of network traffic based on data mining,the overall implementation scheme of the system is designed,and the system is completed based on multi-layer design pattern of MVC.We describe in detail the implementation process of traffic acquisition and analysis module,data storage module,feature extraction engine module and anomaly traffic classification module.Finally,the real network environment is built to verify the feasibility of the system.