Study On Outlier-detection Based Network Anomaly Detection Algorithm

As a reasonable supplementary method to traditional network security technologies,the research of intrusion detection technology has gradually received widespread attentions.However,the data sources of intrusion detection are relatively large.Therefore,applying data mining techniques to discovering intrusion behavioral(or normal behavioral)patterns from massive data has gradually become a key research direction in the field of intrusion detection.As a kind of data mining methods,the outlier detection aims to find data which is much different from most of the data behavioral patterns in the data set.This also coincides with the goal of discovering anomaly data from massive data in intrusion detection.Therefore,using outlier detection to detect intrusions has gradually become an important application area and research direction.Based on the existing network intrusion detection algorithms which are based on outlier detection,this paper extracts the basic framework of outlier-detection based network intrusion detection methods.Then by analyzing existing problems of the framework,the basic framework has been improved.In order to eliminate the false labeled points and reduce the algorithm's dependence on the purity of the training data set,the process of selecting reference samples has been added to the framework.Besides,reference samples selection can reduce the time complexity of the detection process.In order to solve the problem that the clustering number in the selection process of reference samples is difficult to determine,this paper introduces the idea of Natural Neighborhood Graph,and gives a reference sample selection method based on Natural Neighborhood Graph.By using the characteristic of a certain clustering ability of the Natural Neighborhood Graph,the training data set is clustered.Then the clusters are selected as candidate clusters if they meet some conditions we set up,and the reference samples are calculated by the candidate clusters.In view of the problem that the existing anomaly detection method based on outlier detection is difficult to set the anomaly threshold manually,a network anomaly detection method which can adaptively get the anomaly threshold is proposed on the basis of the previous research.The method is named as a network anomaly detection algorithm based on Natural Neighborhood Graph(NAD-NNG).During the process of establishing the normal behavioral profile,the algorithm first adaptively obtains a percentage value ? by using the small clusters' information when selecting the reference samples;then the oversampling idea is used to define outlier scores of the samples.The outlier scores of all the samples in the training data set are calculated and sorted in the descending order,and the |X|×? outlier score is the self-adaptive anomaly threshold.In the anomaly detection stage,the outlier score is calculated through selecting the actual reference sample of the tested sample,thereby effectively reducing the time complexity of the detection phase.To evaluate the effectiveness of the proposed method,this paper chooses the benchmark data set of intrusion detection-KDDCUP99 data set as training and test data set.Also,there are five evaluation metrics selected to measure the proposed method: detection rate,false alarm rate,classification accuracy rate,ROC curve,and AUC value respectively.This paper gives four experiments to show the effectiveness of the proposed method,namely the value selection of ? when selecting reference samples,the influence of oversampling times to detection results,the detection results of new attacks,and the comparison with other three intrusion detection methods.Experiments show that the proposed algorithm can reduce the dependence on parameter selection(especially the selection of anomaly threshold),while still is able to detect intrusions with a lower false alarm rate and a higher detection rate.Besides,the proposed method can effectively detect the new attack types to some extent.