Research On Traffic Sampling For IP Network Measurement

With the rapid development of Information Technology over the past several years,the Internet has had a profound effect on almost every aspect of our society. Personal users can use it to get information, shopping online and entertainments. Business Users can use it to advertise their products to public and realize e-business. Government can use it to improve their service level and administrative efficiency. The dramatic growth of Internet in scale and data link rate for increasingly diverse and demanding purpose brings a new challenge for network measurement. For example, the collection of traffic data from ten thousands of network nodes is essential for that network administrator monitors network performance, but data volumes of 60 byte packet headers on an OC48 link can easily generate 600Gbytes of data in an hour. As a result, the massive resources have to be occupied on storing, transmitting and processing, so network measurement becomes impossible. These challenges motivate the implement of traffic sampling in the large-scale and high-speed next-generation network. The advantages of traffic sampling are to prevent an exhaustion of resources and limit the measurement costs. But, it is well known that sampling distorts the results of network security measurement, network management and performance evaluation because of incomplete data. So, it is important to research how to use the sampling technology in network for accurate results.The research work in this dissertation carries a deep research on traffic sampling in network measurement. The major contents are outlined as follows.In order to reduce the impact of sampled traffic on anomaly detection, a novel method of variable sampling rates in traffic sampling is proposed. By using the hash pattern matching algorithm, we classify incoming packets by flow ID and record their positions. Then, various sampling rates are set according to the decreasing order function of the flow the incoming packet belongs to. Our method increases sampling rates of small flows and resolves the problem that a great many network anomalies are discarded by the random packet sampling because it has a bias towards large flows. Experimental results show that the accuracy of anomaly detection is improved.Recent work has show that network traffic may exhibit properties of Long-Range Dependence (LRD) or self-similarity. In order to prevent inaccurate traffic statistics due to incomplete sampled data, we develop a new sampling method which based on traffic prediction using the FARIMA(Fractal Auto Regression Integrated Moving Average) model. A high sampling rate is employed during periods of peak traffic, and a low sampling rate for periods of low traffic. The analysis results show that our method can generate more accurate traffic measures than systematic sampling and random sampling.A distributed multi-point traffic sampling method that provides an accurate and efficient solution to measure IPv6 traffic is proposed. We use entropy as an evaluation tool to analyze the bit randomness of each byte in IPv6 packet headers, and conclude that the last one byte of Payload Length field and byte number 8,12,14,15,16 of the IPv6 source and destination address fields which have both unchangeability during forwarding and high bit entropy values. Whether a packet is sampled or not based on a hash function computed over the selected bytes. It offers a way to consistently select the same subset of packets at each measurement point, which satisfies the requirement of the distributed multi-point measurement. The advantages of the method is that improved randomness of the sample and the runtime efficiency of the sampling algorithm. Finally, using real IPv6 traffic traces, we prove that the sampled traffic data not only have a good uniformity that satisfies the requirement of sampling randomness, but also can correctly reflect the packet size distribution of full packet trace.A non-intrusive and sampling measurement of one-way delay is described. The main aim of deploying IPv6 networks is to provide QoS guaranteed services, but active (intrusive) measurements that must send test traffic burdening the network impact on the accurate of one-way delay. Non-intrusive measurement method avoids the disadvantage. Traffic sampling based on hash function reduces the amount of measurement data and provides a way to sample the same packet in two different nodes. Moreover, for resolving the clock synchronous problem between sender and receiver during one-way delay measurement, we study how to implement the two kinds of synchronous methods. One is based on GPS receiver and the other is based on linear programming algorithm. The results show that the software method can remove the skew and offset in one-way delay.