Research On High-speed IP Network Flow Measurement Technology

Network traffic measurement is the basic method of getting network behavior, and it’s thepremise of traffic engineering, traffic accounting, security detection and other networkapplications. Internet is developing rapidly, with the constant expansion of network scale and theincreasing of subscribers’ number, the increasing of network bandwidth and the fast-changing ofnetwork topological structure challenge traffic measurement, which needs a more efficientmethod to comprehend the operation status of network and serves optimizing network andimproving network performance.Combined with the research of national fundamental special project for863Program, thisthesis analyzes and summarizes the architecture and the relative techniques of trafficmeasurement of traffic measurement for high-speed network. Aiming at the problem of theexisting network traffic measurement, it focuses on high-speed network data sampling, thesummary storage for flow information and the corresponding techniques of engineeringrealization. Its’ main research work is as follows:For that even random sampling leads to low accuracy of estimation of short flow, a fairsampling algorithm based on Dynamic Count Filter (DCF) is proposed. The algorithm uses themethods that are predetermined measurement error for counting Bloom Filter to count, stratifiedsampling based on flow, protecting extremely short flow and so on, which solves the fairsampling problem of different network flows. By simulation experiments of the actual networkdata, the results show that: it’s easily realized, the measurement error of which is controllable,and it’s space-efficient with low error rate.For the limited computing and storage resources of large flow detection and network datafits heavy-tailed distribution, a method of large flow inspect based on double layer countingBloom Filter (DLCBF) is proposed. DLCBF consists of―dynamic and static‖Counting BloomFilter, uses the separated counting method for large flow and short flow, and employs thepredetermined traffic threshold to detect large flow, at the same time the summary storage oflarge and short flow is completed. By simulation experiments of the actual network data, theresults show that: it’s easily realized, and compared to counting Bloom Filter it saves memoryoverhead.To meet the accuracy and real time demands of high-speed network traffic measurement,this thesis designs a traffic measurement system. It focuses on analyzing concrete realization ofsampling module and storage module, and simulates the system and the results show that it notonly guarantees the balance of various flows but also identifies large flow efficiently.