Research On Application-Oriented IP Flow In High Speed Network

Network traffic monitoring is a fundamental and important technology of network management and operation. Through the network measuring, analyzing and modeling, network operators can deeply understand the composition of network traffic, service growth and change, user behavior as well as quality of service. It plays a significant role in network planning, network operating, QoS assurance and network security.Recently, network applications become more and more complicated, as the booming of emerging applications such as P2P, VoIP, media streaming, instant messaging, online games and so on, ISPs are forced to increase network bandwidth and provide better quality. What’s more, applications dominated by P2P greatly challenge the traditional network monitoring technology.Hiding technology such as dynamic port, port camouflage, proprietary protocol, data encryption, NAT traversal bring great difficulties on traffic classification. The potential security and copyright problem of P2P network, as well as the abuse of bandwidth resources draw the attention of network service providers. To ensure the fairness of the network bandwidth utilization and to protect legitimate copyright, it demands for classification of traffic and action to limit and control them.On the other hand, because of the increasing trend of network channelization, ISPs need to increase investment to expand network capacity, but it is difficult to share the revenue from value-added services. The competition pressure pushes the ISP to focus on the analysis and control of network traffic and to provide differentiated services and marketing.How to identify network traffic and generalize the behavior of users and services efficiently and accurately becomes a hot topic for operators and researchers. The two subjects of this thesis are IP flow monitoring and application identification. IP flow monitoring focuses on the traffic collecting, statistics, reporting and analyzing, help networks operators understand the real-time and historical traffic of network. Application identification focuses on service classification and visualization, and combining with user behavior to monetize the bandwidth.The main contents and innovations of the thesis include the following topics:(1) IP flow monitoring technology of Service and QoS OrientedAfter the study of existing flow-based analysis method based on IP flow record, a new flow concept oriented to service and QoS is presented. The traditional flow technologies represented by Cisco Netflow can only provide the L3 (network layer) and L4 (transport layer) information for network traffic. Due to lack of L7 (application layer) information and performance metrics, they cannot meet the new analysis requirement of network operators, which wants to know all the detail information of the customers, applications and quality of service.In this thesis, a new definition of flow format is proposed. Comparing the traditional technologies, it includes the application category and QoS performance with Netflow compatibility, which meets the demand for deep analysis of users and service. Furthermore, a device which can produce and output this flow record has been designed and implemented.(2) Comprehensive Application Identification Algorithm based on Multiple Flow CharacteristicsOver the past years, many research work of application identification has been proposed, including well-known port, fingerprinting, protocol analysis, and statistical characteristics of flow. Each method has its own advantages and limitations. And application identification is a comprehensive system which needs to utilize whole process of communication and all of the information contained in every flow, every packet and even every byte.This thesis proposed a comprehensive algorithm which integrates DPI, DFI, IP, Port and flow behavior of the above technologies. Multi-flows and single flow information, packet sequence, header fields and payload as well as flow statistical characteristics all need to be considered. Arbitral mechanism based on priority is addressed, which correlate all the factors to identify the application of the IP flow dynamically, and adjust its category continuously according to new packet. It can significantly improve the recognition rate and accuracy. Meanwhile, the implementation method in high-speed network environment is developed and examined.(3) Research on P2P media streaming traffic characteristics base on real traffic data from networkMany researches on traffic characteristics of P2P media streaming have been done in the past. In this thesis, the raw packets are captured and collected from several 10G POS links of backbone layer of Metropolitan Area Network in China. After study of five popular P2P media streaming software, some flow statistics characteristics, such as protocol distribution, flow volume, sequence mode and packet length distribution has been analyzed and modeled. A set of flow attributes were acquired to build a DFI classification model, which can identify P2P media streaming application in high-speed network environment. A method which can extract the attributes is addressed. It is proved that the method can well be used to identify unknown encrypted P2P streaming applications.(4) Traffic monitor system of lOGbps InterfaceBased on new IP flow and classification algorithm, a high-speed traffic monitoring system has been designed and implemented, which can handle bi-direction lOGbps link at wire-speed. It can recognize all kinds of popular applications including dozens of P2P softwares. By deploying probe devices with pure hardware architecture, it can provide various functions of traffic analysis and control, e.g. by user, service, network and flows.Based on the new generation of IP flow record, packet filtering, and token bucket technology, the traffic monitoring system is designed with the distributed, hierarchical and scalable architecture. It has been widely deployed in carrier backbone networks, large amounts of traffic data is collected from different environment, the system can support ISPs to analyze behavior of users and services. Moreover, it can also help them to implement differentiated services and control the abnormal traffic.