Research On Key Technologies Of Intrusion Detection Based On Data Mining

With the rapid development of network and the penetration of network services in daily life,network security issues have emerged in an endless stream,which has different degrees of impact on social stability and economic development.As an important part of information security architecture,intrusion detection collects information from key points in the network system,and analyzes whether there are intrusions in the network.As a process of extracting specific rules and prediction models from massive data,data mining has been widely used in Intrusion Detection System(IDS).Based on the research background and significance,the importance of intrusion detection is expounded first in the paper,and then the research status in the field of intrusion detection is summarized,especially the intrusion detection schemes based on data mining algorithms.The main works are as follows:(1)Aiming at the problem that most data mining algorithms suffer from a high computational burden which makes IDS much less efficient,we propose a classification algorithm based on data clustering and data reduction.In the training stage,the training data are divided into clusters with similar size by Mini Batch K-Means algorithm,meanwhile,the center of each cluster is used as its index.Then,we select representative instances for each cluster to perform the task of data reduction and use the clusters that consist of representative instances to build a K-Nearest Neighbor(KNN)detection model.In the detection stage,we sort clusters according to the distances between the test sample and cluster indexes,and obtain k nearest clusters where we find k nearest neighbors.Experimental results show that searching neighbors by cluster indexes reduces the computational complexity significantly,and classification with reduced data of representative instances not only improves the efficiency,but also maintains high accuracy.(2)Aiming at the problem in traditional Adaboost algorithm that the classification accuracy rate decreases due to the defect of its weight update method,and that the classification speed is low and the computational cost is high which are caused by redundant weak classifier,we propose an Adaboost algorithm based on improved weight update method and selective ensemble.Firstly,at the stage of training weak classifiers,an improved Adaboost algorithm is proposed,which updates the weight of each sample according to its average accuracy of previous t trainings,so that the weights of samples can be updated more evenly.Moreover,this method,to some extent,inhibits the infinite expansion of the weights of noise samples.Secondly,at the stage of combing weak classifiers,a novel similarity measurement between weak classifiers is proposed.And based on this similarity measurement and hierarchical clustering algorithm,the selective ensemble is performed to eliminate redundant weak classifiers.Through this method,the classification speed increases and the computational overhead reduces.Finally,the proposed scheme is simulated and verified based on three data sets: KDDCUP99,waveform and image-segmentation.The experimental results show that the improved Adaboost algorithm can not only improve the classification accuracy and classification speed,but also reduce the computational cost.