Anomaly Traffic Detection Based On Manifold Learning And GBDT

In the era of big data,the traditional method of static rule matching for anomaly Traffic Detection traffic detection can no longer satisfy the changeable and complex network environment.Due to its characteristics of self-learning and self-evolution,machine learning can not only detect anomaly traffic,but also mark and identify unknown anomalies.Therefore,anomaly detection method based on machine learning has become a research hotspot in the field of network security.Since machine learning depends on data processing and the construction of learning models,there are two specific problems to be solved in anomaly traffic detection.The first is to reduce the interference generated by the original data,remove redundant data,and reduce the time overhead of anomaly detection.The second is to need an efficient method to improve the accuracy and precision of anomaly detection.For these two problems,this paper adopts manifold learning in machine learning and GBDT for anomaly detection,and divides the anomaly detection into two parts: data preprocessing and data analysis.The first part is the pre-processing stage,which mainly solves the problem of dimension reduction of network data.The internal structure of network traffic data is complex,the data volume is huge,the variety is various,and there are redundant data.In the process of data mining and information processing,these complex data lead to a huge amount of computation,and reduce the efficiency of analysis and processing.Therefore,it is necessary to reduce the dimension of high-dimensional network data.Based on this,this paper proposes a manifold learning dimension reduction method for supervised discriminant projection.This method performs supervised dimension reduction on the basis of category information.After dimension reduction projection,it can achieve the same type of data aggregation and different types of data dispersion.At the same time,by reducing the dimension of high-dimensional data,the size of data can be effectively reduced,so as to avoid excessive computation.The second part is the data analysis stage,which mainly solves the problem of low classification accuracy of the anomaly detection model.Based on the consideration of different classification costs of different categories of data,this paper proposes a cost-sensitive SGBDT anomaly detection method.This method introduces the cost matrix into the design of the loss function,establishes a cost-sensitive loss function,and reduces the classification cost of different types of data.In order to increase the calculation speed,the idea of random gradient boosting is introduced into the GBDT method.In the decision tree construction of the GBDT method,a part of the training set is randomly selected.By improving the GBDT method,the efficiency of anomaly detection can be effectively improved.Finally,the simulation results show the superiority of the supervised discriminant projection method in data separation.And it proves that the cost-sensitive SGBDT anomaly detection method can effectively improve the anomaly detection.Through the experimental analysis in the pre-processing stage and data analysis stage,it is proved that the method proposed in this paper can effectively improve the accuracy of anomaly detection.