Key Techniques Of Network Flow Measurement

The Internet as one of the most rapidly developing technology in the 20th century, the modern information society has become the most important infrastructure, become the important pillar of national progress and social development. With optical communication technology and the rapid development of broadband wireless communication technology and a new generation of Internet architecture theory research breakthrough, a "bigger, faster, more timely, more convenient, safer, more manageable and more effective" as the main characteristics of a new generation of Internet will come into our work and life. In the rapid development of the Internet brings to our life a lot of convenience and opportunities at the same time, also make us understand and master its operation law is facing new challenges, the original measurement methods and techniques can not meet needs, look for to be able to meet the needs of high-speed network measurement and a scalable network measurement technology become an urgent problem to be solved. In the early part of the network flow measurement, the traditional way is all data measuring, while accurate, but slow processing speed, produce large amount of data, the required storage space is large, and so on. Network traffic sampling techniques to solve the shortcomings of traditional flow measurement methods, made within reasonable error range, meet the needs of the application. However, with the improvement of link rate, packet uniform sampling measurement technology is not accurate fault has been difficult to meet the need. A new solution is according to the specific needs of different applications, targeted treatment choice.In the network flow obey "heavy-tailed feature were discovered, according to the characteristics of flow and packet fair sampling method for identifying algorithm is becoming a hot spot of research.Based on the above analysis of network flow measurement, this paper mainly studies the following aspects:1. First of all, summarize the existing flow identification method; Secondly, on the basis of existing research, choose two more attention-LLR algorithm and Space Saving algorithm, respectively from the principle, analyzes the theoretical analysis of real data using neural network and experimental verification; Finally come to the conclusion that Space Saving Space complexity and time complexity of algorithm and false negative misjudgment rate and other indicators are slightly better than the LLR algorithm. Experimental results showed that the distribution of memory Space is relatively small, PT=0.01%, Space Saving algorithm of false negatives misjudgment rate within a certain range on average is below average false negative LLR algorithm misjudgment rate; And with the increase of memory, two algorithm error rate is smaller, and more and more close, eventually converge to zero.2. Aiming at the existing problem of LLR algorithm, make three improvements, based on an LRU elimination mechanism and stream flow recognition algorithm of combining the summary data structure. By analyzing the complexity of the algorithm and experiment comparison based on the Internet to collect data, the results show that compared with the existing algorithms, the algorithm under the condition of guarantee the accuracy and saves large storage space, improves the algorithm the scalability of the application.3. In view of the existing packet sampling algorithm small flow estimation error big defect, a new packet sampling algorithm is put forward. The algorithm flow according to reach the packet belongs to estimate the size of packet sampling rate, makes the flow contains a packet sampling rate is low, small flow contains packet sampling rate is high. Theoretical analysis and experimental results show that compared with the existing algorithms, this algorithm has higher accuracy and good expansibility, more suitable for engineering application.