Transport and overlay protocols for high-bandwidth applications over the Internet

The Internet has truly been one of the most dominant discoveries over the past two decades. One of the key reasons to its massive success has been the evolution of communication technologies: for instance, long-haul optical links now connect networks across the world and enable communication at high speeds. The increase in network capacities have also given rise to a wide range of high-bandwidth applications such as file sharing and media streaming. For these applications to sustain at such high capacities, the underlying communication protocols should also be utilizing as much of the bandwidth as possible. In this thesis, we propose two end-to-end communication protocols that attain this goal. One is a unicast (one-to-one) transport protocol while the other is an overlay multicast (one-to-many) protocol.Long haul networks suffer from the well-known limitations of TCP over long fat pipes. High-performance protocols like XCP require changes in the network. Other protocols like FastTCP assume nothing about the network but may not perform as well as network-aware protocols. In the first part of my thesis, we present Priority Layered Transport (PLT), a transport protocol that exploits the fact that networks can offer priority queuing, thus finding the sweet spot between assuming too much and too little about the network. Our protocol splits a given transport flow into two prioritized flows. The higher priority flow operates with the legacy congestion control while the lower priority flow aggressively exploits spare capacity in the network while not interfering with the other participating flows. We show through experiments that our protocol can produce near-perfect goodputs even in lossy networks and changing network conditions.The rising popularity of IPTV has triggered renewed interest in overlay multicast. Traditional tree-based solutions are known to be complex, fragile and non-robust to churn and heterogeneity in the end hosts' bandwidth capacities. We present Chunkyspread, an unstructured multicast protocol that uses multiple trees to provide fine-grained control over member load, reacts quickly to membership changes, scales well, and has low overhead. This thesis gives a detailed description of Chunkyspread and an apples-to-apples comparison with the state-of-the-art Splitstream multi-tree algorithm. We show that Chunkyspread exhibits far better control over transmit load than Splitstream, while showing comparable or better latency and responsiveness to churn. This comparison establishes Chunkyspread as the 'best of breed' among tree-based P2P multicast algorithms.