Research On Flow Scheduling For Cloud Data Center Networks

Various applications running in cloud data centers require more strict requirements on network bandwidth and latency than ever.Flow scheduling for data center networks controls in-network flow transmission with the goal to reduce the average flow completion time,and thus optimize user experience,which attracts attention from both academic and industrial worlds in recent years.This dissertation starts from the influence from user tasks of application level to flow scheduling of network level by facing the challenges that lack of flow information such as data generation rates,high overhead introduced by task-aware flow scheduling and scheduling based on job runtime logic.This dissertation investigates on rate-aware flow scheduling,specialized optimizing for tiny tasks and job logic based flow scheduling.The major work and novel contributions are listed as follows:To overcome the challenge that lack of flow information such as data generation rates,this dissertation investigates the complete life cycle from application,system to network of a data flow,proposes a accurate method to measure flow data generation rate and rateaware flow scheduling scheme,named RAX.The basic ideas behind this scheme are that,RAX estimates flow remaining times not only based on the flow remaining sizes but also based on the flow data generation rates;then maps the flows into priority queues supported by commodity switches and performs a MLFQ-like scheduling.Due to flow data generation rate can directly reflect the application demand on networks,RAX avoids blindly flow scheduling.Compared to existing flow scheduling schemes,RAX efficiently utilizes the rich flow information provided by operating systems,including the backlogged data sizes in send buffers and the accurate measurement of application level data generation rates;the prototype of RAX is compatible with legacy TCP/IP stacks and commodity hardware without touching user applications and operating system kernels.RAX is readily deployable and hot-pluggable in running production data centers.The experiment results show that RAX efficiently reduces the average flow completion times,specially for medium size flows.To overcome the challenge that high overhead introduced by task-aware flow scheduling,this dissertation investigates on the low overhead flow scheduling for tiny tasks in data network environment,proposes a light-weight commodity-switch-compatible flow scheduling scheme for tiny tasks,named OPTAS.The basic ideas behind this scheme are that,monitor the network function calls in operating systems and the send buffer status to obtain flow information without touching use applications.To utilize the information from send buffers,OPTAS employs two thresholds,one for backlogged data size,one for data sent size.To avoid network blocked by large tasks and network contention among tiny tasks,OPTAS prioritizes flows of tiny tasks,then performs a FIFO-like scheduling among tiny tasks.Compared to existing task-aware flow scheduling works,OPTAS is low-overhead,quick-responsive and optimized performance for tiny tasks.The experiment results show that OPTAS efficiently reduces the average task completion times,which is an important factor to user experience.To overcome the challenge that schedules flows based on job runtime logic,this dissertation analyses complex job logics,defines a network abstract based on job runtime semantics,named seflow.A seflow contains all flows inside a flow as well as the relationships among them.This dissertation proposes to use job runtime logical bottleneck,LRB,to express job runtime requirement on network;designs a flow scheduling scheme based on job runtime logic,named SLRBF,together with the corresponding flow scheduling system and application programming interfaces.Compared to existing task-aware flow scheduling works,SLRBF algorithm fully explores the logical relationships of intra-job flows,computes logical runtime bottleneck LRB for each job.The jobs are sorted based on their LRB values,while intra-job flows are placed into two priority queues according to whether they are critical to job completion.The experiment results show that SLRBF algorithm efficiently reduces the average job completion times.