Research On Abnormal Network Traffic Detection Based On Ensemble Learning

With the popularity of the Internet,network attacks have become the norm,and abnormal network traffic detection can provide an effective guarantee for intercepting network attacks.To accurately detect abnormal traffic in the network,it is usually necessary to analyze massive amounts of data(high-dimensional data).Analysis of these data not only consumes huge computing resources,reduces the real-time detection,but also reduce the accuracy of detection.To relieve the computational pressure,a variety of feature selection algorithms have emerged.However,the existing feature selection algorithms ignore the relationship between multiple features,leading to inefficient abnormal traffic detection.Due to the effectiveness of machine learning algorithms such as decision trees,machine learning algorithms are increasingly being used to solve the problem of abnormal network traffic detection.However,network abnormal traffic detection based on a single machine learning algorithm has disadvantages such as unstable performance and insufficient deep model generalization ability.This paper first proposes a feature selection algorithm based on voting mechanism.Based on the optimal sub-features obtained by the proposed algorithm,this paper designs an abnormal network traffic detection framework based on ensemble learning.The ensemble learning method combines different learners to a better model,the main contributions are as follows:1.This paper designs an ensemble feature selection algorithm based on voting mechanism to relieve the computational pressure of analyzing high-dimensional data on devices with limited computing resources.This paper uses three filtering schemes: correlation coefficient,chi-square test,and mutual information,and combines two embedding schemes: Random Forest and Light Gradient Boosting Machine(LGBM)to calculate feature contribution to evaluate the optimal sub-features respectively.An ensemble feature selection algorithm based on voting mechanism is designed.The proposed feature selection scheme combines the advantages of the filtering method and the embedding method,which can reduce the calculation amount of the device and reduce the calculation time without affecting or even improving the accuracy of abnormal network traffic detection.2.This paper proposes an abnormal traffic classification algorithm based on Stacking to improve the accuracy of abnormal network traffic detection.The algorithm uses the optimal sub-features obtained by the proposed ensemble feature selection algorithm based on voting mechanism as the training set and trains a Stacking ensemble classifier for abnormal network traffic detection.Specifically,the algorithm handles abnormal network traffic hierarchically.In the first layer,decision trees,gradient boosting decision trees,and multilayer perceptron are used as base classifiers to extract data features to ensure the diversity of the base classifiers;In the second layer,logistic regression is used as the meta-classifier to predict the final prediction result.The proposed abnormal traffic detection algorithm makes up for the low detection efficiency of the traditional single-classifier algorithm.This article uses a cross-validation method to reduce the probability of overfitting in the standard Stacking algorithm.3.To verify the effectiveness of the proposed feature selection and abnormal network traffic detection algorithm,we conducted a series of experiments on the benchmark data set.Experimental results show that the proposed ensemble feature selection algorithm based on voting mechanism can improve detection accuracy by up to 23.25%,while reducing the data volume of the original data set by 67.93%,and reducing the average running time of the device by 25.70%,thereby the proposed algorithm can save computing resources.The proposed Stacking-based abnormal traffic classification algorithm is compared with the three benchmark algorithms of decision tree,gradient boosting decision tree,and multi-layer perceptron,and the accuracy is increased by 2.29%,1.34%,and 2.48%,respectively.