Identifying P2P Traffic And Abnomal Events In The Internet Based On Flow Measurement

Recently, with the popularity of Internet, network measurement has been a basisfor network operation and management. Moreover, traffic measurement is one of themost important parts in the network measurement. However, most of the existing trafficmeasurement systems nowadays can not satisfy the present and future requirements onhigh speed link measurement and new service identification. This thesis aims toimprove the performance of measurement methods and systems. It focuses on some keytechnologies including traffic sampling, P2P traffic identification and control, P2Psuper-node inferring, and abnormal network events detection. The main contributions ofthe thesis are as follows:(1) Flow sampling technology is widely used to mitigate the workload of thenetwork measurement system. During the flow sampling, to reduce the loss of elephantflows will ensure more information contained in the samples and therefore increase theaccuracy of the measurement.. In order to overcome the deficiency of the previousalgorithm hardflow, this thesis presents two improved algorithms called aflow1 andaflow2 which reduce the loss of elephant flows. Compared to the hardflow, the result ofexperiment using Internet traces shows that the loss of elephant flows of improvedalgorithms is less than aflow1 and aflow2 when the sample rate is the same.(2) It is an important to mitigate congestion of the Internet. However, for ameasurement system, identifying and controlling P2P traffic online is a big challenge.This thesis presents a novel method which combines signature matching with randompacket discarding to identify and control the P2P traffic. On the one hand, matching theP2P signatures with Ternary Content Addressable Memory (TCAM), the method canwork online at the links with the speed higher than 2Gbps. On the other hand, referringto the TCP throughput equation, the method controls the throughput of P2P byregulating the discarding rate of P2P packets. The result of the experiment shows thatthis method considering heavy and light workload on the bottleneck link validates themodel. In term of a given discarding rate of P2P packets, the method can control thethroughput of P2P traffic effectively, and in the same time improve the QoS of non-P2Papplications. (3) Though the connection-pattern-based method can identify unknown andencrypted P2P traffic, the previous methods fail to work online at high speed links sincethey have to inspect the IP header multiple times. This thesis presents a method calledInferring P2P's super nodes with packet Sampling (IPS) and solves this puzzle. IPSaims to infer the P2P Super Nodes (In this thesis, a Super Node is specially point to theP2P node to which the number of the connections in the P2P network exceeds athreshold). Firstly, IPS adopts random packets sampling, with sampling ratioself-adapting, to accommodate the requirement of online identification at high speedlink. Secondly, IPS distinguishes the P2P Seed Nodes in packet samples (In this thesis, aSeed Node is specially point to the P2P nodes which are distinguished in packetsamples). Thirdly, originating from the P2P Seed Nodes, IPS distinguishes all the otherP2P nodes in un-sampled packets according to the traffic relationship between the P2Pnodes. Moreover, to save the resources of detecting system, IPS restricts the memoryconsumption through a node replacing algorithm. Finally, experiments using Internettraces have been done and the results have verified that IPS can identify P2P SuperNodes effectively.(4) In the Internet there are often Flash Crowd Events (FCE) and Distributed denialof service attack that Mimic FCE (DMF). Both of them can arose an anomaly of trafficand bring the Web's service performance down. Based on the facts that there aredifferent distributions of source IP addresses between the different anomalies, this thesispresents a method to detect the anomalies in which entropy theory is adopted. The resultof experiments using Internet Web-logs traces and some simulated traces shows that themethod can detect FCE and DMF respectively and accurately.