A Study On Job Management In A Computational Grid Environment

As a rapid developing infrastructure, the grid can share widely distributed computing, storage, data, software, instrument and human resources. It can break through the current computational barriers and coordinate scientific computing and problem solving on large scale. Thereby, it provides a new computational mode for the high performance scientific computing. The grid resources are distributed, heterogeneous and dynamic, which make it a great challenge to research on the grid. In order to improve the usability and QoS of the grid, the job management in the grid is very important, and becomes one of the key research issues in grid computing. However, the job management in the grid environment is yet to be researched systemically.In this dissertation, we have conducted a sophisticated theoretical and technical study on the job management in a computational grid environment, and carries out plentiful experiments. Based on the analysis of job characteristics and requirements of the computational grid, the architecture of job management, job scheduling, job monitoring and performance prediction in the grid environment are mainly investigated. Main innovative contributions of the dissertation are itemized as follows:(1) Proposing a user-oriented job management architecture. Based on the job characteristics and application requirements of the computational grid, the architecture is put forward. Focusing on the job management and aiming to improve the QoS of the grid services, it strives for usability when considering the performance and QoS. The functions include job creating, scheduling, submitting and monitoring, which provided to the users in the mode of workflow. Additionally, several secondary functions are integrated, including visual analysis, performance prediction, checkpoint setting, job migration, job security. The job definition can be reused, and the file operations are automatic and flexible.(2) Proposing and implementing a scheduling strategy and a Heuristic-based Greedy Scheduling Algorithm (HGSA) for various types of jobs. The HGSA uses metric weights of customizable resources and metric workload impact factors as the heuristic knowledge to rank the resources, and uses a greedy algorithm to select the resources. While selecting the resources, the HGSA is able to allocate the job load.(3) Proposing and implementing a job monitoring architecture (MASSIVE Monitoring System), MMS is capable of monitoring the whole process of the job execution. By adopting the distributed and hierarchical structure, this mechanism is able to monitor the jobs running on multiple sites and support multiple users. In addition to basic monitoring of grid jobs, it is also capable of monitoring and steering file operations. A job register table is used to map a job at a client side to its processes on the remote resources. The job register code is used to decide whether a user can access the monitoring information or steer the job. The mechanism supports top-downstart-up of the monitoring components, which can ensure the components at their place.(4) Providing a case and BP neutral network based on a prediction algorithm, CBPP. By utilizing the instructability of the case and the self-learning and excellent ability of nonlinear approximation of the artificial neural network, the CBPP can predict the run time of complicated jobs in the grid environment.(5) Furthermore, a grid job management system has been developed using the works mentioned above. The system hides complicated technical details and provides comprehensive visual steering on the grid job, including job defining, submitting, scheduling, monitoring and analyzing, and result collecting. The system design and implementation are discussed in this dissertation.