Research On The Key Technology Of Data Center Mixed Workload Resource Scheduling

Diverse workload mixing is an important approach to address the problem of low resource utilization rate in data centers.Resource demaind prediction of the online workloads and and the job scheduling of offline workloads are two core technologies of mixed workload resource scheduling.However,existing works the online load resource demand predicting method lacks of intensive investigation on resource use characteristics in the online workload resource prediction and leads the low prediction accuracy and the high time cost.In addition,the job schedulings of offline workloads are conducted in a relatively random manner,which does not consider the impact of the online workload various resource useage and result in the ineffective resource utilization and low job throughput.To this end,an online workload resource prediction method with the consideration of the periodical characteristics of resource use is proposed in this thesis,and then a heuristic offline workload's job scheduling method based on runtime prediction was put forward to further improve the resource utilization in the data centers.The main contributions of this thesis are as follows:(1)An online workload resource prediction method with the consideration of the periodical characteristics of resource use is proposed.In this thesis,based on the analysis on the periodical characteristics of online worlload resource use,the method of autocorrelation function was employed to calculate the online workload resource use period quantitatively,and the resource use sample dataset was divided into several subsequences in accordance with the resource use period.Then,the k-means algorithm was applied to further classify the sub-sequences into two categories: normal sub-sequence and abnormal sub-sequence.Finally,the variation rate of resource use in normal subsequence and abnormal sub-sequences was weighted and integrated to calculate the predicted value of online workload resource usage.(2)A heuristic job scheduling strategy based on the execution time prediction was proposed for offline workloads.Firstly,the data size,memory and CPU resource demand and were selected to model and predict the runtime of the offline workload jobs in the data centers.Secondly,based on the runtime prediction and with the consideration of the variable available resource space formed by the online workload,the simulated annealing heuristic algorithm was adopted to perform the problem modelling and optimization solution for the offline workload job scheduling.(3)The performance evaluations of the proposed methods are demonstratd in this thesis.The experimental results showed that,as compared with the existing online workload resource prediction methods based on ARIMA algorithm,support vector regression algorithm and Markov Model,our proposed online load resource prediction method could reduce the average relative error of prediction by 28.3%,12.3% and 27.4% at maximum.In addition,as the sample data size increased,our method was superior to the comparison object in respect of time consumption.Moreover,as compared with the existing fair scheduling strategy and short job priority scheduling strategy,the offline workload's scheduling strategy proposed in this paper could improve the memory utilization rate,CPU utilization rate and job throughput by 33.2%,16.3% and 42.3% at maximum,respectively.