The Research On Flow-based Network Abnormal Traffic Detection Method

The trend of cybersecurity attacks is no longer direct access to the system and damage to it,but more and more indirect attacks have a negative impact on network traffic.This kind of damage caused by indirect attacks on network traffic can be very serious,and the bandwidth consumption of these attacks will affect the performance of the entire network.Anomaly detection of network traffic can provide good information for network failures and malicious attacks to achieve monitoring and alarming.In addition,with the advent of the era of big data,network traffic has become one of the typical massive data.The ever-increasing amount of data makes manual detection unsustainable.The detection of traditional network abnormal traffic has many limitations.Although humans cannot manually check these huge data streams,machines can.In order to provide a response analysi s to dynamic data sources,this paper proposes that the machine can use streaming-based data analysis methods to filter,highlight and summarize data,and filter and summarize the data before it reaches the user.In order to analyze and detect the abnormality in network traffic,this paper designs a flow-based network abnormal traffic detection system based on data mining and statistical methods.Anomaly detection for DDoS attacks,scanning attacks,and botnets,And study its accuracy,flexibility and efficiency.The main work of the thesis is as follows:(1)This paper designs and implements a streaming network traffic data anomaly detection system,which combines anomaly detection and data analysis to provide a classifier for automatic data classification and an analysis process for explaining data characteristics to users.(2)In order to construct a data classifier in a streaming environment,this paper first uses the distribution estimation to calculat e the distribution of the data.The classifier is then designed to identify anomalous traffic in the network traffic data.Finally,a method of streamlined data sampling based on exponential decay is proposed to realize the automatic classification of streaming network traffic.(3)In order to realize the detection result analysis process in the flow environment,this paper first uses the risk ratio to quantify the possibility that the traffic data points belonging to a specific attribute combination become abnormal v alues.Sec-ondly,using frequent item set mining method to search for attribute combinations with sufficient conditions for analysis,Finally,a method based on hash table-based streaming counter is proposed to dynamically process streaming data.(4)In the analysis of experimental results,a series of real attack data sets are detected to achieve the speed of querying data points above 2M per second,which verifies the effectiveness of the system.Evaluate system performance and verify system scalability.