Research On Mixed-task Flow Scheduling In Cloud Computing Center Network

In recent years,increasing applications rely heavily on the computing services provided by cloud computing center.The mixed-task flows are normally transmitted simultaneously in cloud computing center network,and the quality of service is determined by the transmission efficiency of mixed-task flow.In order to improve computing speed,the distributed computing frameworks are commonly used in existing cloud computing centers with the characteristics of many-to-one transmission mode and mixing flows with different size,leading to the performance limitation and bottleneck of traditional flow abstraction and scheduling method in application,which brings new challenges in the researches of flow scheduling.Coflow is a new abstraction and designed based on the characteristic of parallel computing application in cloud computing center,which has gradually replaced the traditional flow abstraction in recent studies.This paper discusses the current states and progress of Coflow scheduling in cloud computing center,and commonly used Coflow schedulers are introduced in detail,the shortcomings are also analyzed.Some feasible solutions are proposed to overcome the shortcomings of existing Coflow schedulers.The main innovations and contributions of this paper are as follows:1.In order to improve the current situation that the Coflow size prediction methods in the state-of-the-art information-agnostic schedulers are lack of high precision and strong adaptability model,this paper proposes the Coflow size prediction algorithm based on machine learning,which predicts the Coflow size by learning multiple Coflow features in transmission,increasing the accuracy of Coflow size prediction.2.In existing information-agnostic Coflow schedulers,the objective is minimum the average Coflow completion time.However,this paper finds that,because of ignoring the impact of deadline,the Coflow may complete after the deadline,resulting in transmission failure.Therefore,this paper proposes to introduce the feature of urgency weight in scheduling,which can measure whether a Coflow can be completed on time.Also,an urgency weight based information-agnostic Coflow scheduler is presented with goal is maximizing the Coflow completion rate.The simulation results show that compared with the previous Coflow schedulers,the urgency weight based scheduler improves the Coflow completion rate up to 20%.In addition,the Coflow completion time of our design is also shorter.3.By comparing the performance of three kinds of machine learning algorithms in different experimental environments,the Coflow size prediction method is improved.The applicable cloud computing center environments of each machine learning algorithms are also summarized in this paper.4.The widely used Coflow simulator,Coflowsim,which based on Java language,and it is cumbersome to use popular machine learning libraries and frameworks.Also,the simulator is latest updated in 2017 without the most advanced Coflow schedulers.Therefore,this paper rewrites the Coflow simulator by Python language and implements Coflowsim-Python simulator,in which our design and latest Coflow scheduler MCS are deployed.Through Coflowsim-Python,the latest machine learning and artificial intelligence algorithms can be used more conveniently in scheduler design,and the and it is easy to compare with latest Coflow schedulers,also providing convenience for future researchers.