P2P Traffic Detection On Large Scale NetFlow Data

With the increasing use of P2P applications,P2P traffic detection gradually becomes one of the hot topics in network traffic analysis field.The popular P2P applications make more than 50%of the network traffic according to some reports. Since P2P applications can cause network congestion,it becomes an important problem for operators that how can detect P2P traffic out of all the network traffic. P2P applications use random ports to transfer data and P2P system has its own complexity,distribution and variability.All of these facts make P2P detection a hard problem.The main purpose of this paper is to find a way to effectively detect P2P traffic on large scale NetFlow data.All existing P2P detection methods focus on packet data.It is very resource consuming to analyze huge amount of packets over the backbone,so almost all the existing research works can not be put into real use. The current P2P detection products collect packet content by connecting to network in series and use hardware for computing.They have the drawbacks of expensive to deploy,poor extensibility and privacy invasion problem.In this paper, we use NetFlow data for P2P detection and thus we can overcome the above problems.NetFlow data is aggregation and statistics of the packet information.It keeps the important information which indicates the traffic characteristics and makes the data volume smaller.Furthermore,NetFlow technique has been widely used among operators as an industry standard.The main contributions of this paper are:1) Got a series of P2P traffic characteristics according to the way P2P protocols run.For each characteristic,verified its usefulness for differentiating P2P and non-P2P traffic.Chose useful characteristics for detection according to the experimental result.2) Designed a P2P traffic detection algorithm for NetFlow data.The algorithm logically organized the useful characteristics chosen in 1) and made the detection more effective.3) Implemented a P2P traffic detection system INFOPAD based on the algorithm in 2).The system uses database for its data storage and uses SQL queries to implement the detection algorithm,which effectively solves the problem of storing and computing large volume of data.Each detection rule forms an independent module in the system.New rule can be easily integrated into the system as a new module.The system architecture is open and scalable.4) Used real NetFlow data collected from the routers of Shanghai Telecom to test the system.Verified the detection result according to the DPI report from Shanghai Telecom.It is shown that the detection algorithm of INFOPAD system achieves a high accuracy and the system has a satisfactory performance as an offline anaylsis procedure.The detection system implemented in this paper is well applied to P2P traffic detection on backbone networks.The system receives large volume of NetFlow data coming from routers,analyzes the data offline and submits a P2P traffic report at the end.The system has been used in daily network management in Shanghai Telecom.Comparing to DPI products which were already deployed before,our system can achieve almost the same accuracy level.However,it is cheaper to deploy and it is more convenient to maintain and update the detection algorithm by using our system.