Research On Traffic Characterization Of IP Networks With Different Time Granularities

The IP network is a very large scale complex system whose properties are not entirely completely known. Traffic characterization is essential to planning and management of existing IP networks, as well as to designing next generation networks. A huge amount of work has been devoted over the past few years to characterization of the traffic. However, the problem of characterizing traffic of IP network is not one that can be solved easily, once and for all. Complexity and diversity of traffic are constantly growing. With the increment of the size and the bandwidth, the IP network evolution over the past years has been accompanied by the development of various network applications. The nature of the increasing network traffic has taken much more changes. Especially, while there are many researches on characteristics of IP network traffic elsewhere in the world, little has been published about the traffic in China.In this dissertation, we focused on traffic characterization of IP networks with different time granularities in China and concentrated on the following questions. How can we identify individual applications from the mixed traffic? What are the characteristics of the current IP network traffic of China at the application levels? And, what are the influences of the changes or the differences of the application level traffic to the self-similar characteristics of the packet level traffic? Also, this dissertation contains the background material and literature survey, where it provides an overview of pervious works and findings on the measurement and characterization of IP network traffic. The main contributions of the dissertation are mainly comprised of five parts:1) A methodology of online application level traffic identification named multi-phases identification (MPI) is proposed in this dissertation. Identifying and categorizing network traffic by application type is challenging because of the continued evolution of applications, especially of those with a desire to be undetectable. There are several different methods of traffic identification being proposed in recent research for corresponding applications. It is impossible to identify traffic with any one method alone. So in this dissertation we proposed a new methodology combining port-based method, signature-based method and statistical-based method to identify the application level traffic. Experiments shows there are several advantages in MPI:1) these existing traffic identification methods can be easily integrated into MPI to improve the identification accuracy,2) the corresponding new identification method for the new application can be inserted into MPI feasibly with scripts of the identification rule, and3) efficiency of identification can be improved with the mechanism of adaptive justification for the sequence of methods and implemented on multi-CPUs platform. It has been implemented as a part of iCare which is a distributed traffic measurement and management for the high speed network. And we have identified more than90%of IP network traffic from the ISP networks using this proposed method.2) This dissertation took a closer look at the self-similar characteristics from different perspectives which were strongly influenced by the changes and differences of the current traffic in China. Eleven typical Hurst parameter estimated methods were used to investigate if results yielded from earlier experiments are valid for today’s IP networks and how about the self-similarity of application level traffic. In contrast to the previous results which have been widely accepted, this dissertation shows that for the aggregated traffic and the TCP and UDP traffic whether the self-similarity exists is uncertain. Further, break down by the application category, only the traditional and uncategorized traffic are self-similar while the others are not. However, on the view of the individual application of each category, it seems that traffic of every application exhibits self-similarity. To the best of our knowledge, this paper firstly provides the experimental evidence showing that aggregating different groups of self-similar traffic series could generate a traffic series which is either self-similar or non-self-similar. These will help to incorporate more representative assumptions in theoretical and simulation studies, perform more realistic tested experiments, and design benchmarks.3) A detailed and comprehensive traffic characterization of the operational IP networks in China was presented. IP network has taken much more changes. In order to find out how these changes impact on the traffic characteristic, this dissertation made several observations, comparisons and analyses by using the traffic data collected with iCare on an ISP WAN link. From link layer to the application layer, from the inbound perspective to the outbound perspective, and from the packet level to the flow level, this dissertation studied many metrics of traffic which are including composition, contribution and rate variation of bytes, packets and flows. Also this dissertation investigated the correlation properties of the traffic. All results have been compared with real traffic measurements gathered from different IP networks in previous researches. The results show that the traffic continues to grow and compared to the previous analysis there are so many significant differences in the traffic characterization. The differences are not only simply one of traffic volume, but also of traffic mix, protocols, applications, and some other features in terms of bytes, packets, and flows.4) By zooming the granularity of the observation from two different points of view, we analyzes in depth the properties of several real traces collected from two big sites. These traces can be divided into two groups. We use the method of non-parametric test. We find that the Gamma distribution can fit the Internet traffic well when the granularity zoomed is greater than a special value regardless of the observation types. Furthermore, we demonstrate that TCP traffic shows the same behavior with the total traffic and it also has the Gamma distribution characterization, but none of the typical distributions can fit the UDP traffic well in any granularity from any perpective. In particular, our results stay fairly consistent over time and different traces from different sites. Our finding will be help to the research on the traffic modelling and future Internet protocols and architectures.5) An abstract replay method of application’s behavior based on template was proposed. Based on the detailed analysis of most typical replay-methods and frameworks related to the application’s behavior, this dissertation gives a new replay-method based on template which is rely on the systemic classification of all the application’s behaviors. Experiments shows that using this method it could reduce the complexity of building a repay-system of application behavior, especially when adding new applications into the replay-system. More importantly, it is helpful to the application level traffic identification.