Job Scheduling Algorithm In Cloud Environment

As a recent term, cloud computing builds on decades of research in virtualization, distributed computing, utility computing and development of network technology. Cloud computing accelerate the changes of IT industry and promote the development of service-oriented architecture. Through cloud computing, user could obtain on-demand and flexible services, which reduce the user’s cost of access and the expenses of the services providers greatly. MapReduce is a common programming model for cloud computing. Large-scale MapReduce clusters need to deal with the PB level data amount. Job scheduling is an important part of cloud computing technology. Through mapping the job user submit and resource, it could reduce the execution time of the job and increase the throughput of the system. Therefore reasonable job scheduling can improve the utilization of cloud cluster.This article describes the cloud computing concept at first. After analyzing the performance of Min-Min algorithm, we found that the algorithm ignored the load balancing and average completion time. In order to improve the performance of Min-Min algorithm, this paper proposed a Max-D schedule algorithm. By integrating the characteristics of the MapReduce programming model, we improve Max-D algorithm with a resource-job matching table, so that Max-D algorithm can maintain a good performance even in a job varied environment.Hadoop, an open source cloud platform, has been widely research and used.This article describes the key technology of Hadoop platform, and do research of Hadoop job scheduling process and mechanism. FIFO algorithms and fair algorithm is two scheduling algorithms of Hadoop. This paper summed up the applicable scene of each algorithms and proposed a hybrid scheduling strategy based on load monitoring. The core idea of the hybrid scheduling strategy is to select the appropriate scheduling method by cluster load monitoring.Finally, by building a Hadoop experimental environment, we test the performance of the algorithm. Our experiments show that Max-D algorithm is superior to the Min-Min algorithm. Then we using FIFO algorithm, fair scheduling algorithm, Max-D algorithm and hybrid scheduling strategy to schedule both same and varies jobs. The conclusion proved that the hybrid scheduling strategy is in effect in keeping the performance of the cluster.