Simplifying end-to-end network measurement on the Internet

Network measurements play fundamental role in network design and management. However, the sheer scale and diversity of today's Internet make getting high quality measurement results extremely challenging. In this thesis, we describe an approach to simplify planning and executing large-scale Internet measurements. This approach includes means for accessing the entire platform from a local Java application running on a peer computer. We implemented our approach as part of a peer-to-peer platform for Internet measurements called DipZoom (for "Deep Internet Performance Zoom"). Unlike existing approaches that face a difficult challenge of building a measurement platform with sufficiently diverse measurements and measuring hosts, DipZoom implements a matchmaking service instead, which uses peer-to-peer concepts to bring together experimenters in need of measurements with external measurement providers. DipZoom offers easy to use programming interface for conducting large scale Internet measurement experiment and interacting with a large number of measurement providers and it can also be used as a veneer over existing measurement platforms, automating the planning and execution of complex measurements.;Our system was used by ourselves and our colleagues to collect information to conduct various network experiments and to design new protocols. In particular, we utilized our approach to evaluate various techniques for estimating the network latency (often referred to as network distance) between arbitrary Internet hosts. The reason why we investigate the area of network distance estimation is because the modern Internet is experiencing a rapid growth in large-scale distributed applications utilizing overlay and peer-to-peer networks. The performance and scaling properties of these applications crucially depend on forming overlay topologies and exercising communication paths according to network distances between hosts. Besides evaluating existing techniques for network distance estimation, we also propose a novel approach to estimate the network latency between arbitrary Internet hosts in this thesis. We use three landmark hosts forming a triangle in two-dimensional space to estimate the distance between any two arbitrary hosts with simple trigonometrical calculations. To improve the accuracy of estimation, we dynamically choose the "best" triangle for a given pair of hosts using a heuristic algorithm. Our experiments show that this approach achieves an order of magnitude or more improvement in computational efficiency over the classic landmarks-based approach while producing more accurate estimates. We have also implemented a prototype application for real-time CDN server selection using our dynamic triangle selection approach, which has demonstrated the practicality of using our approach in real world applications. Although still in its preliminary stage, our application already showed promising results.