A Decentralized Grid Resource Discovery Scheme Based On Resource Classification

Grid is a hot topic in distributed computing nowadays. Just like Web, which is developed to meet the requirement of sharing scientific experimental datum at the beginning, Grid concept is motivated by problems in modern scientific research to share high-performance computer, large database and expensive instrument. The real and specific problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations. Considerable progress has been made on the construction of such an infrastructure since 1990s, and the Grid concept has been widely accepted and refined.Resource discovery is a basic problem in Grid resource management, which concerns discovering resources in Grid to meet the requirement of applications. Compared with traditional distributed systems, Grid aims to integrate much more resources of varieties of types belonging to different organizations, hi addition, providers and consumers of Grid resources usually have different or even contradictory interests. With further development of Grid, centralized Grid resource discovery schemes have potential scalability and security problems. Therefore, in order to survive the dynamic and larger-scale Grid environment, resource discovery should be decentralized and should not rely on centralized coordination and control.Resource discovery schemes in current Grid system are basically centralized. Though protocols are proposed to support interactions between different information nodes, there is no specification on how multiple information nodes are organized to provide a general-purpose decentralized resource discovery scheme. On the other hand, the efficiency of decentralized resource discovery remains to be an open problem in dynamic and large-scale Grid environment.Focusing on scalable and efficient decentralized resource discovery in Grid environment, this paper makes following contributions:1. Proposing and designing a decentralized resource discovery scheme based on resource classificationBased on the technique in file-sharing P2P networks of grouping nodes sharing similar files to improve efficiency, this paper suggests a resource discovery scheme based on resource classification, which is called RC. In this scheme, Grid information nodes with the same type of registered resources are self-organized to communities, and a DHT (Distributed Hash Tables) based bootstrap network is constructed to navigate between different communities. Theoretical models on decentralized resource discovery are provided, under which RC is compared with existing schemes theoretically. By controlling propagation of messages into related communities, RC improves time andnetwork efficiency at the cost of topological maintenance overhead.2. Proposing and analyzing a load-balance approach based on rumor mongering to avoid hotspots in bootstrap network.DHT P2P based Bootstrap network is an important component in implementation of RC, which enables self-organization of community and efficient message forwarding between different communities. Therefore, load balance on bootstrap network is an important issue. A load balance approach is put forward, which makes use of the periodical topological maintenance communications between nodes in DHT P2P network to piggyback and propagate popular information in hotspots. The effectiveness of this approach is studied with Chord protocols. Results show that, by propagating popular information on bootstrap network in background, the load on hotspot can be effectively reduced and little communication overhead is introduced.3. Proposing and analyzing a greedy-search based resource location inside communityRC does not specify detailed protocols on request forwarding inside communities. A Greedy search based resource location method called GSBRL is put forward to control the results returned to users, under which only resources with relative high qualities among all qualified candidates within a community are returned. An analysis model is built to study the effectiveness of GSBRL, and results are also validated by simulation. It is shown that, this method can find resources with relative high qualities among all qualified resources in a big community with small number of hops. The overhead is very low, and the effectiveness can be further improved when integrated with other random message forwarding strategies.4. Simulation study of RCA simulation environment is constructed to study decentralized Grid resource discovery, and a simulation engine is also implemented. RC is compared with other decentralized schemes by extensive simulations. Results show that, RC has much smaller time and network overhead than those schemes propagating requests or resource information update randomly. In addition, RC significantly reduces network overhead compared with those random grouping schemes, though it may have a little bigger time overhead in the worst case. The increment of topological maintenance in RC is proved to be acceptable and controllable.Two application cases of RC in volunteer computing and simulation grid are discussed, and further theoretical problems are also pointed out.