The Comprehensive Sample Processing Method In Network Intrusion Detection

Intrusion detection, as a very effective and important active safety-defense technology, has been a frontier topic studied by research scholars. The quality of the data directly affects the accuracy, effectiveness and scalability of the classification model, which further affects the performance of the entire system. The available training data detected from network is massive and unbalanced, which challenges the real-time and accuracy problem of intrusion detection. Therefore, an efficient comprehensive treatment of the sample is necessary before the classification model of intrusion detection is built.The Special network environment requires unusual pre-procession. The known class distribution can’t be used in unbalanced treatment in data mining directly, because that the data is created in the network continuous; excessive number of samples leads to trouble for the compression process, as well as the unbalanced data leads to accuracy problem. So the pre-procession for network data must be a combined treatment.In this paper, we preprocessed data in two aspects as follows:(1) Use the K-S statistics irrelevant to class distribution to segment data set, to reduce the unbalance level of each data subset and reduce the corresponding effect on classification rules. The experimental result indicates that the accuracy and efficiency of the unbalanced data classification algorithm can be improved by this method.(2) Improve Affinity Propagation cluster algorithm, let samples which is close to the cluster center link to the center without clustering, so only part of data will be clustered, which will reduce the time and space cost greatly. And the model will be adjusted by the result of link, to accurate the clustering results. The experimental result indicates that this method can reduce the space-time cost of cluster algorithm, and keep good compression results.Finally, we try to seek an integrated approach combined with unbalanced data procession and sample data compression, design a pre-procession method independent of the classification, and design a lightweight network-security intrusion detection model. To determine the effectiveness of the proposed method, it is applied to dataset KDD99with different classification method to test the applicability of the model. The experimental result indicates that the time efficiency and accuracy of detecting unknown sample can be improved by the proposed model with3classification methods. The proposed model has practical availability for that it can deal with big data set which requires low time and space, and can choose one of the three classification method according to the demand.