Ensemble Feature Selection For Minority Class And Intrusion Detection

Network security technology has been a focus until now. As an indispensable part of the network security system, the intrusion detection system has been widely applied in many fields. It can monitor the network status in real time, and alarms the intrusion activities without delay. It makes up the deficiency in firewall.As of now there are many researchers depth study the intrusion detection, and many effective algorithms have been proposed. However, with the rapid development of the internet, the network packets gradually become larger and more complex. The intrusions hide in these vast amounts of network packets, and it’s difficult to be detected. Furthermore, the intrusions are small pieces, comparing with the amount of the normal network traffic. Traditional intrusion detection algorithms disregard the small amount of intrusions, and it is no longer suitable for the new network environment, while they take the overall detection accuracy as testing standards. On the other hand, most of the existing detection methods are supervised and they consider the class label of the sample data. And there are few unsupervised algorithms. For the vast amounts of network data sets, the supervised algorithms greatly increase the cost of the intrusion detection system, and reduce the operational efficiency of the system. Therefore, it is quite necessary to develop new methods to solve these problems.This paper presents two solutions for this problem, one is ensemble feature selection and the other is improved resampling. In the imbalance problems, the difficulty for feature selection is that it’s difficult to determine the features that can distinguish the small samples and the major samples. The paper considers integrated technology and unsupervised feature selection method comprehensively, and proposes a selection method based on ensemble technology. The experiments show that the method is able to select the characteristics that contribute to distinguish the small samples from the major samples, and to maintain high efficiency of intrusion detection.In order to improve the detection efficiency of the few samples in the intrusion detection system, an unsupervised method that based on sampling and support vector clustering algorithm (SVCR) is raised. The method combines support vector clustering and resampling. It under-samples the majority class, over-samples the minority class and chooses the samples to follow up learning carefully. The results show that there is a substantial improvement in the detection efficiency for the small samples.