Research On Prediction Of Vasp Job Execution Time For Backfilling Optimization

In recent years,the demand for high-performance computing in various fields is increasing day by day.Service providers often need to invest a lot of money in hard-ware resources to meet the computing needs.However,the resource utilization of most platforms is not very high,so the effective utilization of platform resources has become an urgent problem.As we all know,the scheduling strategy is the central link between the upper layer operations and the underlying hardware resources,and it is a factor that has a significant impact on resource utilization and user experience.At present,most high-performance computing clusters in China usually adopt traditional job scheduling methods such as first-come,first-serve,because such scheduling methods are simple and practical.However,in order to ensure the principle of fairness,it is often necessary to reserve resources for a job with a large resource demand and which cannot be satisfied by the left resource,which is prone to the emergence of lots of idle resource fragments.In response to the above problems,a more common strategy is to use backfilling,which optimizes the use of resource fragments.However,good backfilling depends on the prediction of job execution time,and backfilling scheduling is difficult to apply to pro-duction systems due to the lack of job estimation execution time,so the prediction of job execution time is also a problem that needs to be solved urgently.Given that vasp is one of the most popular high-performance computing applica-tions in China and accounts for about 43%of all jobs and about 46%of the machine time used,Therefore,it is of great significance to predict the execution time of vasp jobs.This article analyzes the characteristics of the vasp job and extracts the corresponding job feature set by parsing the log and the input file,and we proposed a add-on learning algorithm named IRPA which based on stacking model to predict the execution time of the vasp jobs.the algorithm performs add-on training based on the prediction results of multiple submodels,and combines the advantages of each sub-prediction model to achieve higher prediction accuracy.Afterwards,a prediction algorithm based on Ra-dial Basis Function(RBF)Networks which named BRBF is proposed and the algorithm mainly uses the advantage of the radial basis network to fit the unknown nonlinear func-tion.We tested the IRPA and BRBF algorithms using the real vasp job dataset on TC4600 platform and the comparison of experimental results with several other basic methods shows that IRPA and BRBF have higher prediction performance at coarse granular-ity.Finally,we combine prediction result for vasp jobs of the BRBF algorithm with backfilling scheduling,and use the real workload on the TC4600 platform to simulate.The experimental results are compared with several benchmark methods,which further demonstrate the value and significance of our work.