A framework and toolkit for the effective measurement and representation of Internet internal characteristics

Interest in building compact, accurate and efficient end-to-end network models increases as network-aware applications and services over the Internet are deployed. These models could be used to optimize network resources utilization, improve the quality of service of content delivery and help analyze network performance. They are particularly important for popular servers driving a large portion of the Internet traffic. This thesis describes a framework for building such models.; The main contributions of this thesis are as follows. First, this thesis presents a framework for Metric Induced Network Topologies MINT: a general framework that uses correlations between end-to-end measurements across multiple flows emanating from a single host to model network properties between this host and the flows' endpoints. A salient feature of this framework is its ability to compress or parameterize the representation of the network with respect to sensitivity parameters that define a desired level of details. Second, this thesis characterizes a broad class of metrics (e.g. loss rate, delay) for which MINT is applicable. Third, this thesis introduces new unicast end-to-end active probing techniques that enable the correlation of observations collected from endpoints and new techniques that are capable of estimating network properties along arbitrary path segments. Fourth, this thesis instantiates MINT for a variety of metrics—namely loss, delay, and bandwidth—in the context of unicast messaging. Fifth, this thesis provides mechanisms for the integration of network models (snapshots) obtained at different points in time and/or from different hosts. Finally, this thesis presents an implementation of the proposed techniques in the Linux kernel through a set of Application Programming Interface API called PERISCOPE.; The thesis contributions comprise analytical results, simulation and Internet deployment. Theoretical correctness of the MINT framework constructions is established through vigorous analysis. Extensive simulation experiments show the effectiveness of our approach and its robustness over a wide range of simulated network conditions. Internet deployment verifies and validates PERISCOPE usability over wide area networks.; The utility of the MINT framework instantiations can be appreciated through its deployment in a variety of network-aware applications and services. Examples include Server Selection, Overlay Network Organization, Admission Control, Flow Scheduling and Cache/Replica Placement.