Research On Coflow Scheduling For Fairness And Efficiency In Data Center Network

With the development of cloud computing and the arrival of the era of big data,various cluster applications in data centers emerge endlessly.The network data transmission has replaced storage and computing as a bottleneck in application performance.At the network transport layer,Coflow(a collection of flows with common goals)can better describe the semantic abstraction generated by a set of data transfer between cluster servers.Recently,the research on Coflow mainly focuses on:1)how to improve the efficiency of Coflow(i.e,minimize Coflow completion time),2)how to ensure Coflow performance isolation(i.e,fairness).With coflow size information,previous solutions perform well in simultaneously achieving fairness and fast completion for coflows.However,as coflow size is difficult to obtain in practice,scheduling coflows without prior information is more research-oriented.Despite the prevalence,existing information-agnostic solutions either solely focus on fairness or fast completion.In this paper,we propose an information-agnostic scheduler OSTB,which achieve both fairness and fast completion.The core ideas of OSTB are: 1)utilize the idea of long-term isolation guarantee to achieve fairness,2)use port occupancy ratio to predict Coflow sizes,3)combine improved multi-level feedback queue(MLFQ)with D-CLAS degradation strategy to classify Coflows and make order decisions.4)use the bandwidth multiplexing strategy,which give the smaller coflows more opportunities to send data,while still guaranteeing that the wider or larger coflows have available bandwidth all the time.Extensive experiments were conducted based on Facebook's real data set,demonstrating that OSTB incurs 59% fewer delayed coflows than NC-DRF when both of them use the isolation-optimal solution(e.g,DRF)as baseline.OSTB provide a better isolation guarantee.Compared to the most efficient scheduling Aalo,OSTB is 1.2×faster and improves efficiency.In addition,when the parameters fluctuate within the appropriate range,the OSTB performance is almost unaffected and has stability.