Research On Key Technologies Of Network Intrusion Detection Based On Data Mining

In recent decades, the explosive growth of the Internet has resulted in an increasing number of people using the Internet in their daily life and business. However, risks from network attacks cause damage to social stability and economic development. As one of main techniques for protecting network security, intrusion detection technology can detect network attacks before they induce widespread damage and provide important basis for establishing defense strategy. With the continuous expansion of the network, a variety of new security vulnerabilities and network attacks are emerging, which put forward a higher requirement for the performance of intrusion detection system (IDS).Data mining is an intelligence analysis technique, which is able to discover the useful knowledge from amount of data. This paper reviews the latest development achieved in the field of intrusion detection based on data mining, and focus on the key technologies of network intrusion detection based on data mining. Specifically, this paper studies some key issues concerning the research on intrusion detection based on data mining, which are feature reduction, sample reduction, the anomaly detection method based on outlier mining technology and hybrid intrusion detection model. The main works of this paper can be summarized as follows:(1)Research on the feature reduction technology in intrusion detection and designs a feature extraction method that is able to adapt the application of intrusion detection. Feature reduction, including feature selection and feature extraction, can reduce the dimensionality of feature vectors to make many data mining algorithms get better results. In this paper, by analyzing the related research on feature reduction methods for intrusion detection, a feature extraction method based on the distance sum of cluster centers is proposed and applied to intrusion detection. This method uses a specific relationship between each sample and the cluster centers——distance sum, to convert the original feature vector of data from a high-dimensional feature space into a low-dimensional space. The experiments in this paper show that this feature extraction method is effective in intrusion detection.(2)Research on the sample reduction technology in intrusion detection and designs a sample reduction method that is able to adapt the application of intrusion detection. Sample reduction is one of the ways for data reduction and can be used to reduce the size of data set. Compared with using data mining technologies to the entire original data set, using a subset of the original data set selected by sample reduction method can reduce the cost of data mining and speed up the data mining process, and sometimes even able to achieve better result. In order to select a subset of high quality from the original data set, this paper proposes a stratified sample method based on class’s centroid. In this method, a new concept is introduced to measure the representative power of an instance with respect to its class in a given data set, and a strategy is proposed to divide the data set into subsets of equal size. A subset can be selected from the original training data set and will be used as training data to build intrusion detection model. Experiments show that this sample reduction method is effective in intrusion detection.(3)Research on outlier mining technology in intrusion detection and designs an anomaly detection method based on outlier mining. Outlier mining technology can discover the outliers from data set. By analyzing the related research on outlier mining methods applied in the field of intrusion detection, this paper proposes an anomaly detection method based on the change of cluster centers. This anomaly detection method extracts reference samples (cluster centers) by using clustering algorithm on a set of normal instances, and applys the effect of adding or not adding a target sample (can be the training instance or the instance to be detected) on the variation of cluster centers to give a "score of outlierness" for this target instance, and an instance to be detected will be identified as an abnormal instance if its score of outlierness is above an anomaly threshold. The experiments show that this anomaly detection method can detect network anomalies with a high detection rate.(4)Research on the structure of hybrid intrusion detection model and designs a two-level hybrid intrusion detection model that composes of three detection modules. Hybrid intrusion detection model integrates both misuse detection and anomaly detection methods, so it may combine the advantages of both technologies. Based on the analysis of the structures and the advantages and disadvantages for the existing several types of hybrid intrusion detection model, this paper proposes a two-level hybrid intrusion detection model composed of two anomaly detection modules and one misuse detection module. The detection modules in the two stages of this hybrid intrusion detection model work well together, and the two detection modules in the stage2can identify the false positives and false negatives generated by the detection module in the stage1, respectively. The experiments show that this hybrid intrusion detection model is able to detect network anomalies with a low false alarm rate and a high detection rate.