Offline Network Traffic Analysis

Network traffic analysis and classification is a fundamental technology enabling network operators to monitor network usage and manage network effectively. Currently, network bandwidth is occupied by large volumes of unprofitable P2P traffic, which causes severe security and quality of service problems, and puts the network operators in the dilemma of repeatedly "congestion-construction-congestion" circle. The solution to the above problem calls for the differentiations of network users and applications, and provide customizable services. Thus, for network operators, the research on sensing network applications, and building a manageable and harmonious network, are becoming increasingly important and urgent.The thesis presents the research on network traffic classification technology and system, including the following topics:Offline network traffic analysis and classification system, design and implementation:An analysis tool is needed to analyze the characteristics of the network traffic, for subsequential network classification. Current network traffic measurement tools do not analyze the statistical features of large volumes of network traffic to be used for classification, and evaluate the performance of various traffic classification algorithms. Thus we design and implement an offline network traffic analysis and classification system for such application. The system focus on the analysis of flow level statistical features of network traffic, and is capable of analyzing the characteristics of flow feature distribution for different applications, and presenting the results through a graphical user interface. Moreover, the system supports multiple classification algorithms expecially based on statistical flow features, and can be used as a teaching and research platform for the evaluation of the performance different classification algorithms for different applications under different network environments. The performance of the algorithms and solutions proposed in this thesis are evaluated with the system presented in this chapter. Network flow classification based on flow statistical features: Current classification methods either use individual flow features, or social features. We use statistical features of both individual flow and their social context for network traffic classification. Based on the analysis conducted with tools described in chapter three, we propose a group of linearly separable features that are easy to extract. We use multi-logistic classification on these features, achieving both reduced complexity and improved recognition rate for P2P traffic, as compared with existing approaches. In addition, current network traffic classification algorithms are sensitive to the quality of training data, and perform well only when the majority of flows in the training data are correctly labeled. We proposed a multi-variable decision tree based on adaptive hierarchical clustering, to solve the problem. The experiments show that the algorithm can significantly improve recall with minor impact to accuracy, and correctly identify flows not recognized by payload signature methods in the training dataset. The experiments are conducted on the evaluation platform as described in chapter three.Online multi-strategic network traffic classification, research and improvement:Existing research on online network traffic classification based on flow features suffer from problems of simplistic assumptions, limited network classes, and low efficiency. We analyze the characteristics of network traffic from prospect of protocol fingerprint distribution, short/long flow distribution, protocol and port distribution, and propose an online multi-strategic network traffic classification framework taking into account the above characteristics. The framework applies multiple classification strategies to different flows, to reduce system overhead and achieve higher efficiency and ensure online timely classification of network traffic. The offline system in chapter three is improved with the design paradigm in this chapter. And finally, we evaluate the performance of classifying long UDP flows based on incomplete flow features.