Internet Measurement And Traffic Characterization Based On Application Layer And Transport Layer

The Internet continually evolves in scope and complexity, much faster than our ability of observing and characterizing Internet, even predicting its development. As a major and critical means of understanding Internet, the study of Internet measurement is challenged.Firstly, the state of the art in Internet measurement still lacks long-term measurement and charactization studies, and these studies are important for revealing the evolvement of Internet traffic or network applications which can be used to evaluate the impact of new technologies or protocols and predict the network development; thus it is an important issue valuable for researching. Secondly, simple and small-scale measurement is hard to handle the growing complexity and scope of Internet. How can the comprehensive scheme be designed for concrete measurement purposes? How can the measurement scale be enlarged? They are both crucial issues in measurement methodology. Thirdly, the variety of datasets used does not allow systematic comparison of methods, and their poor comparability of results is further amplified by the lack of standardized measures and classifications goals. The forth, the research conclusions are presented singly and abstractly which hinders the communications among researchers and make difficulties for people to under Internet intuitively. How can a measurement conclusion be expressed lively? It is an issue worthy of discussion. In addition, the theoretical study on Internet measurement is backward currently.Towards the problems mentioned above, this paper has conducted various measurement and analysis studies, focusing on traffic characterization and traffic classification. The main research and innovation of this thesis are as follows:1) Considering the lack of network evolvement description and the oneness and small scale of measurement means, we conduct several measurements and analysis on Internet applications. By carrying out four annual large-scale data collection events over the top layer of the DNS hierarchy, this long-term study of the DNS root system shows some trends, including the consistency of query types distribution, workload, misconfigured pollution, etc. We also present new results on security-related attributes of the client population:a short-term measure to improve DNS security, and a surprising decreasing trend in the fraction of DNSSec-capable clients. Our insights on IPv6data are limited to the nodes that collected IPv6traffic, which does show growth. On the other hand, this thesis also conducted measurements on two popular P2P applications, for understanding the behavior of fake servers in eDonkey system, and exploring Xunlei's architecture and characterizing its downloading process, even analyzing bandwidth theft in Xunlei.2) Considering the lack of the study of multiple datasets, this thesis characterizes the features of the UDP flow along with the recent rising of P2P file-sharing applications. Moreover, an across-study on two campus traces is done both from the transport layer and application layer. We also evaluate the features on flow level over different datasets. Our experiment results show that the packet size is a robust feature for applying the traffic classification model.3) In the view of the simple and dull way to present datasets, this thesis has designed and deployed an interactive data visualization platform by AJAX, which not only presents the analysis results comprehensively, but also supports the interactive between users and datasets flexible.4) Considering the backward of Internet measurement theoretical study, this thesis has presented a structured taxonomy of traffic classification papers and their datasets, for revealing issues and challenges in traffic classification. On the other hand, based on our measurement and analysis studies, the discussion of methods and technique is open for conducting global trace collection experiments and utilizing collected various datasets, in order to contribute some insights to the establishment of the measurement infrastructure over our campus.