Research On High-speed Network Traffic Measurement

With the application of the Internet have been constantly expanded and deepened, the rapid growth of network traffic, increasingly complex network structure, network protocol, species diversity. The abilities of network situational awareness and meticulous network management are needed in order to ensure the efficient running of the Internet. To achieve the above goal, fine-grained analysis of the network traffic is the fundamental method, and it is of great significance in optimizing network resources and improving the performance of network services..This paper focuses on challenging problems of network traffic measurement. Based on embedded understanding of network traffic measurement models, we study on technologies of network traffic measurement. The main work is as follows:First, to solve the problem of the current network traffic measurement methods suffering overflow of packets counter, a new data structure, called Counting Bloom Filter structure(DOFC-BF structure), is proposed for high-speed network traffic. The complexity and space complexity of DOFC-BF is fully analyzed and its performance is proved through real data experiments.Second, a big-small stream distinguish method based on Bloom Filter is proposed. This method estimates parameters of Pareto distribution using Hill estimation, in order to get perfect parameters for flow size estimation. After flows are properly distinguished, diversely sample methods can be chosen for flows of different sizes, so as to achieve effective traffic reduction. A series of experiments show that our method has high performance while ensuring certain accuracy.Finally, we implement a network measurement prototype on multi-core network processor, in which DOFC-BF structure and optimized storage solution are adapted to summarize high-speed network traffic. Real network traffic is used in experiments which proves the satisfying performance of the prototype and proposed structures.