The Analysis And Detection Of Abnormal Flows Based On Data Mining

Along with the rapid development of Internet technology and the increasing types of services,the data of Internet is increasing gradually.On one hand,the development of the Internet brings great convenience to people.On the other hand,it increases the probabilities of occurrences of abnormal situations.There are important practical significance and value for us to study the method how to detect abnormal situations accurately and quickly in Internet traffic and to make reasonable and effective response.In recent years,researchers have proposed the method to detect abnormal situations based on data mining,which is to discover implicit and useful knowledge from massive data in order to form the detection rules.Then scholars quickly conducted extensive research in this field.First,this paper conducts extensive research of the detection and analysis technology of abnormal traffic at home and abroad.Therefore,we have some knowledge of the development and status of detection and analysis technology.Then we provide an overview of abnormal traffic and its classification and conduct a detailed analysis and comparison of the common traffic detection and abnormal traffic detection technology.According to their principles,their advantages and disadvantages are described in the following paragraphs.Secondly,this paper conducts extensive research of clustering algorithm in the data mining field and then uses DBSCAN algorithm based on density to detect abnormal traffic.We do training and testing on the offline data using the improved DBSCAN clustering algorithm based on grid to acquire the characteristics and tendency of abnormal flows and to distinguish what is normal and what is abnormal.This method can find clusters in any shape or different sizes and identify border points and remove noise points effectively.It's in this way that the clustering results are more accurate,in other words,the detection accuracy improved.Then the detection model will be applied to the real time traffic.Thirdly,this paper conducts extensive research of the methods of classifying abnormal traffic.We use cross entropy theory to measure the distribution changes of the flow characteristics.When there are abnormal behaviors,the cross entropy will increase suddenly between two consecutive observation points.We use the cross entropy of SrcIP?DestIP?SrcPort?DestPort?InDegree?OutDegree?FSD,PKTS to classify abnormal traffic.As the previous paper shows,we define the property characteristics of five common abnormal flows,which are Worm,DoS Attacks,DDoS Attacks,Port Scanning Attacks,Abnormal P2P traffic.This method is to classify the abnormal traffic by the characteristics of them,using Euclidean distance to judge the types of attacks,so that the accuracy of the analysis results is improved.Finally,we set the model of detecting and analyzing abnormal traffic using the offline data set KDD 99 with the improved DBSCAN clustering algorithm based on grid and cross entropy theory.And we use the model to detect and analyze the real time flow,which is collected in the form of NetFlow.Our research can provide detection basis for future research of locating abnormal traffic quickly,identifying the causes of abnormal situations and providing solutions.