Distributed Real-time P2P Detection System On Research And Implementation

In recent years, the demand for backbone Internet bandwidth has been explosive growth at 75-125% per year, driven by the growing broadband Internet users and growing use of "bandwidth-hungry" applications. In addition to web type applications, Peer-to-Peer (P2P) applications are the main contributors, including P2P download and P2P streaming media. To ensure the fairness of network resource utilization,identify and control the traffic of P2P applications has become the focus of traffic identification. However, with the popularization of 10 Gigabit Ethernet, common networkmonitoring systems and intrusion detection systems, has become more difficult to face the monitor requirement of high-speed network environment.Our experience shows that the passive packet capture performance of a low-end server, are unable to adapt to 1-Gigabit environments, because CPU and bus bandwidth has become a bottleneck. Estimated from the existing 100Mbps based detection system, the system bottleneck is more likely to cause by the efficiency of algorithms, rather than the hardware factors. Although the P2P detection and classification algorithm are now coming in thick and fast, but most of the efficiency of behavior-based P2P detection algorithms are only stay in the "laboratory stage".Through the previous research based on passive packet capture of gigabit environment, we found a better solution and propose a distributed behavior-based P2P detection system which can provide wire-speed packet capture of 10Gbps or greater environment without using any special hardware. Then describes the key point in data processing of high traffic environment: filtering, and compare the existing traffic filtering method, we propose a novel node-based traffic filtering algorithm to identify the dynamic feature of nodes, in order to reducing the duplication of identification issues in known node.In this paper, while improve the recognition accuracy of behavior-based P2P detection algorithms, we also propose a novel method to improve the efficiency of P2P identification, by reducing memory footprint and choose a new statistical approach. Then we propose real-time algorithms for behavior-based P2P identification.Based on the above method, we design and implementation of distributed real-time system for behavior-based P2P identification, through a novel way for monitoring higher speed interfaces (e.g. 10-Gigabit) by distributing their traffic across a group of lower speed sensors (e.g. 1-Gigabit). Our system can be divided into three layers: the interface layer, the logic layer and the data aggregation layer. In the implementation chapter, we explained technology including lock-free ring buffer, shared memory for inter process, optimization of busy waiting, node/flow hash algorithms, and so on, to improve efficiency of the distributed system. Prototype system has been put into the actual network, and its identification accuracy is over 95%.