Study On The Class Imbalance Problem In Network Intrusion Detection System

With the continuous expansion of network scope and scale,the threat of network intrusion is more serious than ever.Network intrusion detection system(NIDS)is a key security device for detecting malicious intrusion activities in modern networks.It can detect abnormal network activities in time and issue alarms,so that administrators can take countermeasures as soon as possible.With the rapid increase of network traffic data and the continuous emergence of zero-day vulnerability attacks,network intrusion detection systems based on machine learning and deep learning based on anomaly detection have received more and more attention and research.However,in the intrusion detection data set,the distribution of normal class samples and attack class samples is imbalanced.This kind of imbalance problem limits the recognition performance of the classifier for rare attacks.To improve the detection rate of minority classes while ensuring efficiency,this paper studies the class imbalance problem in the NIDS.Based on the existing classic class imbalance processing technology,combined with the characteristics of modern network intrusion detection datasets,a new class imbalance processing technology is proposed.Based on class imbalance processing technology,ML and DL,around the key technology of intrusion detection optimization strategy and intelligent intrusion recognition,two effective network intrusion detection systems are proposed.The main research contents of this paper are summarized as follows.(1)From the data level,an effective comprehensive sampling method SGM is proposed.First,use the Synthetic Minority Over-Sampling Technique(SMOTE)to oversampling the minority samples.Then use Gaussian mixture model(GMM)to perform cluster-based under-sampling for majority samples,and finally balance all classes of data.And propose a network intrusion detection model SGM-CNN that combines imbalance processing technology SGM and one-dimensional Convolutional Neural Network(CNN).On the UNSW-NB15 data set and the CICIDS2017 data set,by comparing with the other five types of imbalance processing methods and two classification algorithms,the following conclusion can be drawn: the SGM-CNN model provides an effective solution for class imbalance intrusion detection.(2)From the model construction level,a two-stage fine-grained network intrusion detection model combining Light GBM algorithm and CNN is proposed.The first stage uses the Light GBM algorithm to identify normal and abnormal in network traffic,and the second stage uses a CNN to perform fine-grained attack category detection on the samples predicted to be anomalies in the first stage.We verified the superiority of the two-stage model on the current relatively new CSE-CIC-IDS2018 data set.The experimental results show that the two-stage intrusion detection model not only makes full use of large-scale data,but also improves detection accuracy and efficiency,and can adapt to imbalanced large-scale network flow data,which is superior to existing advanced researchers.