Network Traffic Monitoring Model Based On The Sampling Algorithm And Automatic Clustering Algorithm

NMS's intuitionism and real-time processing speed are the major problems, NetFlow technology makes the implementation of NMS which is based on the "flow" more facilitate. With the support of a series of well-known manufacturers to NetFlow technology, how to analyse network traffic and find the unconventional use of network with the help of network data collected by NetFlow, have become a new researching direction of network management.In this paper, we describe thoroughly the uniform sampling, systematic sampling, adaptive random sampling, stratified time packet sampling and threshold sampling etc. We profoundly analyze the principle and realization way of clustering algorithms based on automatic inferring patterns, which includes one-dimensional and multidimensional clustering algorithms. The method of traffic characterization can automatically group the traffics into minimal clusters, and dynamically generate clusters to match the application problem. For example, other than reporting five hundred small flows, or the amount of TCP traffic to port 80, or the "top ten hosts", this method can reveal a certain percent of traffic which are used by TCP connections between clients and a particular group of Web servers. We analyze and modify open-source tools such as SoftFlowd, Fprobe, Flowd, etc. And we apply them to the structure of experimental environments. Finally, we combine the sampling algorithm and automatic clustering algorithm, to improve the monitoring system model based on clustering algorithm and sampling algorithm. And we also give their theoretical analysis. Combining the sampling algorithm, automatic clustering algorithm can be used for a new model of automatic classification such as the classification of network worms or peer-to-peer applications in the case of the flow structure is unknown. Additionaly, we implement the prototype system design and its application in some practical network.The combination method of the sampling algorithm and the automatic clustering algorithm can not only adapt to the current network using situation, but also reduce greatly the computation time, memory share.Any abnormal use of network traffic has its own flow characteristics. Thus, finding these characteristics from the frequent occurrence of network data stream is the key of NMS. When we adopt the above new algorithm, we can more easily find the abnormal pheomena. We also apply our new algorithm to network model of a university campus network. The test data seems promising.