Research On High-speed Network Traffic Measurement And Analysis And Its Applications

Network traffic measurement is an important task in network management.However,with the rapid development of the Internet,network traffic has exploded.The measurement under high-speed network traffic faces great challenges.Sketch is a method that can store and summarize data streams.It can measure and query data streams.It has several typical Sketch algorithms: Count-Min Sketch,CU Sketch,and Count Sketch,which can be applied to the network.Come to the measurement.However,due to the characteristics of the network traffic itself,when using Sketch for measurement,a lot of space may be wasted,resulting in problems such as low space utilization.In addition,because Sketch uses a hash function to summarize data streams,hash collisions can cause errors in estimated values,especially for small flows.Therefore,the work of this paper is mainly aimed at improving the algorithm of network traffic measurement for the two problems of low space utilization and overestimation of small traffic.Firstly,this study introduces the idea of carry,and proposes a structure that combines multiple Sketch with Counting Bloom Filter(CBF)—Self-Adaption Sketch(SA Sketch).This structure can dynamically apply for space and create Sketches based on the size of the network traffic to be measured,and use CBF to store the number of Sketches used by the current traffic,thereby improving space utilization.Experimental results show that the error of SA Sketch in point query is lower than that of count-min Sketch,CU Sketch and Count Sketch.With heavy hitter detection,the accuracy is also greatly improved.When SA Sketch achieves the same measurement accuracy as other Sketches,SA Sketch uses less space and maintains the same level of throughput as other Sketch algorithms.When the load factor is larger,the accuracy of SA Sketch is more obvious.Subsequent research found that because the Counting Bloom Filter uses hash functions to store and query elements,there are certain errors in the query.Therefore,when performing point query on traffic in SA Sketch,the number of Sketches in the query may be wrong,resulting in a large error in the final query value.In order to solve this problem,this study uses the idea of cuckoo hash,combines the improved cuckoo hash table with Sketch,and proposes Cuckoo-Based Self-Adaption Sketch(CBSA Sketch).This structure uses the improved cuckoo hash table to store the number of Sketches,so as to accurately query the number of Sketches and further improve the accuracy of point queries.Experimental results show that the accuracy of CBSA Sketch is significantly improved compared with count-min Sketch,CU Sketch and Count Sketch in point query and heavy hitter detection.With the same accuracy,the CBSA Sketch improves average throughput and effectively saves memory space.In summary,the Cuckoo-Based Self-Adaption Sketch proposed in this study effectively improves space utilization and measurement accuracy,and improves throughput to a certain extent.As the network load factor is larger,the improvement is more obvious than other algorithms.Therefore,it is more suitable for high-speed network traffic measurement,and can choose the optimal parameters according to the size of the network traffic and the purpose of the measurement to process the data stream.