Research And Implementation Of Heterogeneous Computing Cluster Scheduling System

The compute-intensive computing tasks have the following characteristics: 1. large-scale, long computation time; 2. Different types of computing tasks need different computing power. To deal with these computational tasks requires the system has certain reliability, fault tolerance, scalability.Nowadays,there are many parallel programming model, but they have their scope of application and shortcomings.This paper builds up a heterogeneous computing cluster system to accelerate the process of massive specific applications. The system consists of three parts: management node, scheduling node and computing nodes. The management node is mainly responsible for the management and monitoring of tasks. The scheduling node is responsible for task scheduling and distribution of computing task. And the computing node processes tasks with the powerful computing capabilities of the computing devices. This paper mainly researches the task assignment algorithm, the reduction algorithm of result and the re-calculation method of missed task, so that to achieve the fault tolerance and scalability for the cluster.Finally, this paper tests the availability and performance of our system in large-scale cluster and small-scale cluster environment. The results show that, in large-scale cluster, the efficiency of our system can reach more than 80%, besides, the system can effectively support heterogeneous computing nodes.