Job Scheduling Algorithm Based On Hadoop Platform Optimization Research

In recent years, cloud computing technology with the development of computers, academic research and application in industrial development is also increasing, from here, a lot of cloud computing system was put into practice. Cloud computing systems are generally based on Hadoop platform to develop and debug the program. Hadoop is an open source framework, the framework is a large database applications running on the cluster, parallel processing is the most outstanding technical performance Hadoop, which is relatively transparent to the developer, allows developers to develop as usual, without the need to consider the question of parallel computing, parallel processing platform automatically. However, the late time of the Hadoop platform, development time is very short, requires further study.This paper focuses on job scheduling technologies, this technology is a core technology of Hadoop platform. Job scheduling technologies studied include two aspects: one is the job how to correctly order; second, the rational allocation of resources and effective control. These two aspects directly influences the whole performance and the utilization rate of resources.In this article, through reading a large number of related literature and analyzing the existing algorithms, found that there are some disadvantages.So, improve the performance of Hadoop platform and resource utilization is very necessary.This article main research content is as follows:First, research on cloud computing and Hadoop platform was introduced, in-depth understanding of a Hadoop application background, insight into how Hadoop architecture, has studied the related algorithms.Then, the Hadoop platform job scheduling algorithm is researched, focuses on three commonly used algorithms are: FIFO scheduling algorithm for fair scheduling algorithms and calculation algorithms. And analyze the advantages and disadvantages of three kinds of algorithms, focusing on the in-depth study of fair scheduling algorithm and its deficiency is found, and a brief description.Finally, deficiencies of fair scheduling algorithm, I put forward for the improvement of traditional fair scheduling algorithm, namely delay fair scheduling algorithm based on priority. Choose accurate benchmark tests, the comparative tests, analyze results, finally delay fair scheduling algorithm based on priority can improve overall system performance.