Resource Evaluation Model For Grid Scheduling System

Grid is a new generation technology of network computing and application built upon Internet. It connects various computers, large-scale databases, sensors, remote instruments and other resources, which are geographically distributed and heterogeneous, via high speed network to establish a virtual high-performance computing environment that is relative transparent to users, and provide more resources, functions and services for technologists and common people.Grid can solve large-scale problems of science, engineering and business by choosing, sharing and aggregating the resources that are geographically distributed and heterogeneous. In the process of large-scale problems solving, the grid resources used change dynamically. Computing Grid needs real-time, precise, dynamical performance information of resources for scheduling system to make decisions in the dynamical environment of grid. Therefore, it is very important for load management system to choose appropriate resources from dynamical varied resource pool. Considering the characteristics of grid resources and evaluating the resources reasonably can support scheduling system to complete scheduling policies, make the use of grid resources more reasonably and improve the performance of grid implications.This paper is part of project"Real-time monitoring and predicting of grid applications'execution performance", which is funded by National Natural Science Foundation. Through the research and analysis of the characteristics of grid resources and grid scheduling system, a Resource Evaluation Model for grid scheduling system is designed based on the grid infrastructure Globus Toolkit 4. It includes Cluster resource information collecting from local scheduler; using grid information service component MDS4 to support the collection of resource information by dealing withaggregation, subscription/notification, update of the resource information spanning the domains and searching Cluster resource information through information querying interface of MDS4; improving the scheduling plug-in framework of meta-scheduler CSF4 reasonably; presenting resource evaluation arithmetic should suit specific scheduling policy and integrating resource evaluation arithmetic into the meta-scheduler for scheduling system to make decisions on matching tasks and resources.Grid resources have the characteristics different from LAN resources or machine resources. The analysis of the characteristics has been carried on in this paper. As a further abstract of grid resources, services, expensive equipments, specific applications and others all have been defined as resources. Both types and functions diversity of the resources in previous systems can't compare with grid resources. Thus, grid resources have some own characteristics, including 1) Heterogeneity, the local resource management system in each grid node may be different, resource protocols they adopt may be different, and even local resource management policies may be different; 2) Dynamic, the availability, load and capacity of resources are likely to be dynamic; 3) Autonomy, grid resources have their own local management institutions or in the charge of local management institutions, grid resources have local autonomy ability strongly or weakly .The goal of grid scheduling system is to organize the idle WAN resources for the users who haven't enough resources, assign the applications to the heterogeneous computing nodes and get optimal or near-optimal performance. Scheduling system need to solve the problems such as resource using authority, resource state information, work management, scheduling policies and others. How to consider the characteristics of grid resources, evaluate the resources reasonably and support scheduling system to make decisions is necessary for scheduling system to complete scheduling policies efficiently and reasonably. In gridenvironment, the users of resources usually aren't the owners and grid scheduling systems don't have the full authority of resources. Grid scheduling system dispatches the jobs to local scheduler which is real resource owner first, and then the jobs are scheduled by local scheduler. Therefore, the goal to collect information from is set for local scheduler but not the resources charged by local scheduler. The definitions of resources have been given according to local scheduler, including concrete statistic information of cluster machine resources, abstract statistic information of cluster and management policy information.According to the characteristics of grid resources, a cluster resource information provider getClrInfo has been implemented for local scheduler LSF based on MDS4. Grid Information Service has been added between scheduling system and information collection module. Information Grid technology provides hierarchical aggregation, subscription, notification and index for grid resource information. Monitoring and Discovery service MDS4 is a component in Globus Toolkit 4. It mainly contains a series of network services based on monitoring and discovery mechanism and provides a set of tools and implication interfaces for discovering, publishing and accessing various resources information in grid.Because the resource evaluation depends on the content and policy of evaluation strongly, in this paper, resource evaluation module and grid scheduling system have been integrated not only to make the content and policy of evaluation more clear, but also to support scheduling system to make decisions conveniently and directly. The scheduling plug-in framework of meta-scheduler CSF4 has been improved reasonably and added in the resource filter and evaluation interfaces to make resource evaluation arithmetic suit for special scheduling policy. In addition, based on the resource evaluation module, a scheduling arithmetic with resource evaluation has been designed and implemented.Finally, in the simulative grid environment, the scheduling arithmetic with resource evaluation has been experimented and compared with FCFS Round-robin scheduling policy. As a result, resource evaluation arithmetic can consider local scheduling policy sufficiently and evaluate the state of cluster resources precisely. Comparing with primary FCFS Round-robin scheduling policy in CSF4, scheduling system with resource evaluation arithmetic can dispatch jobs according to the real state of cluster resources, make the utilization of grid resources more reasonable, reduce the total job completion time and improve the whole efficiency of the system.In the future, we should make further research on the types of resource information which are collected by cluster resource information provider, and use the information to optimize scheduling policies. Moreover, we will analyze different grid application characteristics and design resource evaluation arithmetic according to the requirements from applications and users.