Sampling Technique Network Measurement

With the construction and development of the new generation of the Internet, networkbehavior becomes very complex, abnormal attack against the network has become more serious,this status threatened network management and safety to a certain extent. Network measurementis the basis of network performance analysis and modeling, it plays an increasingly importantrole in network management. However, the data of the high-speed network is large, so storing ormeasuring all the flow or packet information has become impossible. And the rate of networktraffic has many uncertain factors in a single period of time, and the rate of these mutationsbrings excessive consumption of system resources. The introducing of sampling techniquessolves the problem’s performance bottlenecks successfully, becoming one of the focus of theresearch of network traffic project.In this thesis, we introduced the network traffic measurement and analysis techniques, anddescribed the difficulties encountered in measuring the high-speed network, noting the importantrole of sampling techniques in the network measurement. Then detailed overview of the contentsof sampling techniques discussed several common sampling methods, systematic andcomprehensive analysis of the key technologies and important algorithms associated with thesampling measurement, such as the Bloom filter and timeout strategies. Finally, by studying thecurrent network characteristics, this thesis proposed a new sampling algorithm by combiningsampling techniques and Bloom filter algorithms and dynamic timeout strategy to measure theflow measurement. Among these, Bloom filter is simple, and can find and match resourcequickly; timeout strategy as a sign of judgment stream output, plays an important role in theaccuracy of flow characteristics and utilization of the stream cache. By performance analysis andsimulation experiments, we prove that the proposed algorithm can not only ensure measurementaccuracy, but also improve the resource utilization of resources. The specific research contentsare given as follow:(1) In this thesis, we studied the Bloom filter algorithm and improved CBF algorithmdeeply, and designed a dynamic counting bloom filter (DCBF) algorithm for the reason thatcurrent CBF algorithm will result in counter overflow with a high flow. The algorithm uses amulti-CBF which can add new adaptive CBF when the flow is large, and prevent the CBFmeasurement error caused by overflow. We Simulation experiments conducted by the CBF andFCBF algorithm based sampling algorithm compares the measurement error terms, the analysisshows that the proposed algorithm improves the accuracy of the sample, reducing the spaceutilization. We compared the measurement deviations of both our algorithm and FCBF sampling algorithm through simulation experiments, so we know the proposed algorithm can improves theaccuracy of the sample, and reduce the space utilization.(2) With the continuous expansion of the network size, network traffic characteristicsbecome more complex and hard to predict, static sampling methods cannot meet the high-speednetwork measurement. We presents an adaptive flow sampling algorithm, which stratifiespackets using time and using a fixed maximum number of samples in the layer, so you can keepthe measuring accuracy with a small network traffic and keep the resources in control with asurge in traffic. Then, we use the two-layers self-adaptive timeout (TSAT) policy to control theoutput stream for the defects exist in network measurement applications with a fixed time-outstrategy. TSAT policy uses a two-stream space to maintain separate space for single packetstream flow, and uses a smaller timeout. By comparing this algorithm and the algorithm based onNetFlow sampling, we verify that the algorithm has self-adaptive, high accuracy and resourcecontrollability.