Rapid generation of structural model from network measurements

The usefulness of simulations and analysis heavily depends on good models of network traffic. However, it is difficult to model and simulate Internet traffic because of the network's great heterogeneity and rapid change. The statistical properties at any given point in the Internet often change over time, and traffic also varies from location to location.;While a trace-driven modeling approach can generally provide a good fit of the measured traffic, previous studies in this area typically treat measured traffic as a time-series of statistics and focus on capturing the statistical characteristics of empirical data to model network traffic. Such open-loop trace-replay approaches, however, ignore the fact that feedback-based traffic like TCP tends to react to network's properties such as the level of congestion. Additionally, such approaches provide little or no insight about the observed characteristics of measured traffic and its underlying causes.;Additionally, in order to get a complete picture of network-wide view of the traffic, it is necessary to integrate data collected from multiple points in a network. While there is increasing interest in using high-level SNMP data from many routers for traffic matrix estimation, currently detailed traffic models are generated based on traces from a single tap into the network. In today's production IP network, however, it is typically infeasible to collect fine-grained, packet-level information at every single router in a large network due to administrative and technical issues. Even if it were economically feasible to synchronize and monitor every router in a large network, such massive amounts of data would place enormous demands on storage and computation resources for large ISPs with many links to monitor.;In this dissertation, we develop new approaches and tools that support rapid generation of realistic traffic models from live network measurements. Specifically, based on the structural modeling approach, we develop a tool that utilizes measured traffic to estimate end-user behavior and network conditions and generate trace-driven application-level simulation models. To reduce the huge overhead from continuously collecting measurements on every single link, we propose a methodology to infer traffic based on the correlations between similar networks.;To demonstrate our approaches, we first develop a structural model of UDP-based RealAudio traffic and evaluate its accuracy via simulation. Additionally, we show the use of multi-scaling analytic techniques for debugging and validating the model. Based on the structural modeling approach, we then develop a tool called RAMP that automates and integrates the process from collecting traces to ultimately generating structural simulation models for web and FTP traffic. We show the utilities of RAMP using three immediate applications: near-real-time trace-driven simulation, generation of high bandwidth synthetic network traces, and analysis and modeling of malicious traffic.;Finally, we present a methodology to infer traffic by exploring the similarity of user populations across different networks. The inferred traffic statistics are then incorporated into RAMP to generate realistic traffic models for places where measurements are not available. We demonstrate and validate our approach using a case study on traces of web traffic and evaluate its effectiveness via analysis and simulation.