The Design And Implementation Of A Job Pre-scheduling Model For The Compute Cluster

High Performance Cluster(HPC) is often used in the field of computing of the large-scalecomputing, for its good scalability and reliability, besides its powerful computing ability. Forexample, the industry model designed by hyperworks needs to deal with complex and precisedata, which only in the cluster,can it get the result in a reasonable time in many cases. TheJob scheduler is the key part of the whole cluster system. Usually, people put their strength inhow to improve the internal job scheduler of the cluster, this paper proposes a job"pre-schedule" model, which implements the job scheduler in the HPC service platformsystem, it can improve the quality of service and efficiency. HPC service platform sorts jobsaccording to certain pre-scheduling strategy, and then submits to the compute cluster,meanwhile it monitors all states of jobs during jobs are in the process of running.HPC service platform is a interface between users and the compute cluster, it avoidsinputting complex commands for user, and brings great convenience to users for using thecompute cluster. However, jobs may be interrupted in the process of running due to manyunpredictable situations. In tradition, users ought to re-submit jobs manually, which greatlyimpair cluster’s efficiency and Qos. Apparently, it lacks the ability to handle these situationswith the original job schedule method, which is the key point studied in this paper. Usually,people put their strength in how to improve the internal job scheduler of the cluster, and paylittle attention to problem of the service quality of the compute cluster, this paper proposes ajob "pre-schedule" model, it will help user for the fine-grained job management.HPC serviceplatform helps users to submit jobs to the compute cluster, and it also defines many job statesto label the different running stages of jobs, through identifying jobs’ states, it helps users toavoid re-submit jobs after a running error arises and handles it automatically.Job scheduling algorithm is an important aspect of the job pre-scheduling model for thecompute cluster, the job pre-scheduling is different from the job scheduling, since it isassociated with the management strategy of compute cluster, but it can design thepre-scheduling algorithm refer to the job scheduling algorithm. In this paper, thepre-scheduler uses Greedy algorithm as its main pre-scheduling strategy for the jobpre-scheduling.This paper implements a job pre-scheduling model for the compute cluster, including thedefinition and implementation of state model, the communication between highperformance computing service platform and the computing cluster, the control managementof job preset scheduler, job scheduling algorithm of design and implementation, model management and finally the whole test of high performance computing service platform.