Algorithms for explicit coordination to prevent network congestion in data center Ethernet

Large cluster-based cloud computing platforms increasingly use commodity Ethernet technologies, such as Gigabit Ethernet, 10GigE, and Fibre Channel over Ethernet (FCoE), for intra-cluster communication. Traffic congestion can become a performance concern in such an Ethernet-based network fabric due to consolidation of multiple virtual machines (VMs) onto fewer physical nodes as well as consolidation of data and storage traffic over a common layer-2 network. Heavy network traffic from competing sources can severely degrade the performance of communication-intensive applications in the form of throughput collapse and increased network latencies. We make the case that explicit coordination of network transmission activities can significantly improve network performance and help reduce or even eliminate network congestion in cloud computing clusters. XCo, eXplicit Coordination, is a software-only solution executing only in the end-nodes, that is fully transparent to applications, presently deployable, and complementary to any switch-level hardware support. We have implemented this explicit coordination logic in NS3(Network Simulator 3) to evaluate our results against large network setups and various topologies. In this thesis we present the design, implementation and evaluation of our system. This thesis focuses mainly on the design and development of central controller that explicitly schedules the network bandwidth to avoid congestion. Our evaluation results show that explicit coordination can alleviate congestion-driven performance problems experienced by data center Ethernet. The results show that with explicit coordination we are able to utilize the network bandwidth to its full capacity. NS3 evaluations also show that explicit coordination logic can be scaled to large network setups.