Research On Traffic Load Balancing Mechanism For Data Center Networks

With the development of cloud computing and big data processing techniques,the scale of data center has become larger and larger,and the performance and number of its servers are increasing exponentially.The main portion of traffic has changed from flowing from data center to end users to residing within the data center.Data center runs a variety of workloads simultaneously,ranging from short-lived latency-sensitive small flows to long-lived bandwidth-hungry large flows.Data centers run a variety of services simultaneous,resulting in the following traffic characteristics: most flows are short in size;however a small fraction of the flows with high speed and large amount of bytes consume most of the network bandwidth.Flow transmission rate changes frequently,which increases the burstiness and unpredictability of the traffic matrix.In data center networks,some links are underutilized,while others are congested frequently.Network congestion leads to the increase of transport latency and the degradation of network throughput,even dropping packets,which impair the services performance and QoS.Fat-tree topologies have been widely adopted by constructing data center networks.In these data center networks,there are multiple parallel paths existing between a pair of servers connected to different edge switches.In data center networks,based on the Fat-Tree topologies and traffic characteristics,it is important to evenly balance traffic load across multiple diverse parallel paths,which facilitates high throughput,low latency network transmission and meets the services transport demands.On the basis of summarizing the research background and current results of data center networks,this paper analyzes the importance of traffic balancing mechanism in improving data center network performance.According to the Fat-Tree topology and traffic characteristics,the key techniques of network traffic balancing have been studied at the aim of alleviating network congestion and optimizing network performance.A Centralized-Distributed Traffic Load Balancing(CDT-LB)mechanism is proposed,which according to the detection and classification of data center network traffic,combines the centralized flow scheduling with distributed traffic load balancing.CDT-LB outperforms the distributed scheme in terms of the network performance.In the process of research and design of CDT-LB mechanism,this paper makes three main contributions,which are large flow detection algorithm based on low-rate flow eviction and D-Left Hash,distributed traffic balancing based on self-timing Flowlet and flow scheduling with minimum path shifting.Data center network is a kind of high speed network and has similar traffic characteristics with others.Based on the research of flow detection and classification in data center,a novel dual-threshold algorithm is proposed for large flow detection,which combines d-Left Hash table with the eviction of low-rate flows' entries in order to identify large flows efficiently.The data structure of flow entries is indexed by d-Left Hash table to meet the performance requirements of high speed packet processing.A theoretical analysis is conducted to demonstrate the accuracy,performance and memory overhead of the proposed detection algorithm.Experimental results on real data sets show that the proposed algorithm outperforms L-LRU algorithms in terms of accuracy and performance at comparable memory overhead.During the process of researching the distributed traffic load balancing mechanism,a novel traffic splitting algorithm named Self-Timing Flowlet(ST-Flowlet)is proposed.Traffic splitting is a vital technique for traffic load balancing,which is a challenging problem because of the tradeoff between achieving well-distributed effects and avoiding out-of-order packets.Three main approaches exist for splitting traffic.The first is packet-based splitting,which independently assigns each packet to a path.The second is flow-based splitting,which assigns all packets of a flow to the same path.The third is Flowlet-based splitting.Flowlet is burst of same flow's consecutive packets featured by packets arrival interval,which is a fixed time interval or time gap measured by ping messages.The fine granularity of traffic splitting achieves well-distributed effect,but causes high probability of packet reordering.According to the variations of arrival interval of packets which belong to same flow,ST-Flowlet partitions individual flow across multiple parallel paths in order to quickly response to network congestion and effectively improve the network transmission performance based on the tradeoff between balanced effect and packet reordering.During the designing of centralized-distributed traffic balancing mechanism,a large flow scheduling scheme with minimal number of path shifting is proposed.On the one hand,reducing the switching of large flows' paths will decrease the probability of packets reordering and alleviate the performance degradation caused by out-of-order packets;on the other hand,it lowers the impact on the traffic of the target links,so as to improve the transport stability of the total traffic in the network.In order to validate the centralized-distributed traffic load balancing mechanism(CDT-LB),a packet-level simulation system based on NS3 platform is developed.A series of simulation experiments show that the proposed CDT-LB mechanism can effectively improve the performance of data center network.The distributed traffic load balancing scheme based on Self-Timing Flowlet outperforms LetFlow,Drill and Presto in terms of improving network performance.The network performance can be further optimized by combining ST-Flowlet,LetFlow or other distributed traffic balancing schemes with centralized large-flow scheduling.CDT-LB achieves high throughput and low latency of network transmission,by combining centralized scheduling of large flows with distributed balancing of other flows.CDT-LB has good scalability and can be applied to large-scale data center network.The research results of this paper provide an important reference for the development and deployment of traffic balancing mechanism in the data center network.