Resource Demand Prediction And Convergence Determination Of Computational Jobs

With the development of cloud computing and the advent of the era of big data,the demand for big data services is growing.The computing speed and storage capacity required for big data analysis make the combination of cloud computing and big data increasingly close.At the same time,the elasticity of cloud computing can be a good solution to the peak problem in the field of high-performance computing.The combina-tion of high-performance computing and cloud computing is increasingly attracting the attention of the industry.The rise of the industrial cloud era is inevitable,many types of programs,including high-performance computing(HPC)are moving to the cloud.Computing tasks gradually occupy an increasingly important position in the cloud data center.Today,the low resource utilization rate of data centers is still a problem that needs to be solved urgently.Even a top company like Google,its data center resource uti-lization is less than 50%.The main reason is that the user overestimates the resource requirements of the task,and the service provider needs to guarantee the quality of ser-vice,resulting in a large amount of computing resources being idle.Moreover,there are some scientific computing tasks in the endless loop,whose calculation results are worthless for users,but they still occupy computing resources.To improve resource utilization and user experience in data centers,in this paper,we design an adaptive dynamic resource demand forecasting algorithm for computa-tional tasks.Cloud resource managers dynamically scale resources and improve re-source utilization through predictive values.Using the monitoring data of Google Data Center,we verify the accuracy and adaptability of our algorithm on the production plat-form.Furthermore,we design a two-stage model for the convergence determination of the iterative computing task VASP in HPC.Its purpose is to determine the convergence of the user's submitted tasks so as to end the non-convergence long tasks in advance and improve resource utilization and user experience.Through the monitoring data of VASP tasks in the China University of Science and Technology Supercomputer Center,we verify the effectiveness of the algorithm.