Research On Coflow Scheduling Mechanism In The Data Center Networks

The development of network technology and the rapid growth of social economy provide a huge impetus for the construction of data center,and the application services provided by data center are increasing.The huge amount of data generated by application service brings great pressure to the data center,many applications use distributed storage scheme to store data.In this scenario,how to guarantee the transmission requirements of applications and how to improve the network performance have attracted the attention of relevant researchers.Most cluster applications in data center are scheduled in the unit of Coflow.Coflow is defined as a set of data flows with correlation between two groups of servers caused by parallel computing.The traditional strategy of scheduling single flow model focuses on satisfying the transmission demand of a single stream to optimize network performance,but the single flow model is not suitable for Coflow scheduling.There are still many problems and bottlenecks in the traffic scheduling in the data center,such as: the transmission of tasks cannot be guaranteed within the deadline,and the completion time of tasks is too long.Based on the existing research results of traffic scheduling,this thesis studies the Coflow scheduling mechanism,and the specific research contents and contributions are as follows:(1)In order to meet the transmission requirements of time sensitive Coflow in data center,this dissertation designs a centralized Coflow scheduling mechanism based on information awareness High Priority Satisfied First(HPSF).The thesis analyzes the process of data flow generation in parallel computing,and introduces the Coflow data flow model.Then this dissertation models the Coflow generation process with known information,and designs a node placement algorithm to assign priority to Coflow and a Coflow scheduling mechanism to guarantee the transmission requirements of time sensitive Coflow.The scheduling mechanism satisfies the Coflow transmission rate according to the priority order,so that as many Coflows as possible can complete the transmission within the deadline.In this way,we can ensure that Coflow only takes up the least bandwidth resources when resources are sufficient,so as to reduce the contention for bandwidth resources between data streams;in the case of resource shortage,we can ensure the transmission requirements of high priority Coflow.The simulation results show that,compared with the classic Varys and DCS scheduling mechanism,the HPSF mechanism can increase the number of Coflows to 83% and 79% within the time limit under the condition of sufficient and scarce resources.(2)In order to solve the problem of flow scheduling in cluster computing framework,the scheduling strategy based on Coflow has become a research hot spot.A Coflow is a collection of data flows between two different stages of the same parallel computing task.Coflow scheduling in the case of unknown prior information depends on the data flow information of the sent part to infer the data size of Coflow and allocate the scheduling sequence for Coflow,which is easy to cause congestion.In this thesis,we design an effective Coflow scheduling mechanism namely,Classification According to Ports Number(CAPN).In the mechanism,firstly,Coflows are quickly classified according to the Few Ports Number Scheduling First(FPSF)algorithm,and then Coflows with different priorities are scheduled and adjusted,which greatly reduce the average Coflow completion time(CCT).Simulation results show that compared with the classical Aalo and MCS scheduling mechanisms,our CAPN mechanism can reduce the completion time of Coflow by by31.32% and 25.72%,respectively.