Researching On Dynamic Adaptive Job Scheduling Algorithm Based On Hadoop Platform

Cloud computing, a kind of new calculation model, based on Internet technology is changing people’s understanding of network computing. It solves the need for huge amounts of data storage and handling. Cloud computing can be thought of as the core of the change of a new generation of information technology in the future. It can be regarded as the third wave of the IT industry.This paper selects Hadoop Cloud Computing Platform as the research object. Now a research focus in the Hadoop platform is job scheduling and resource allocation problems, they relate to the overall performance and resource utilization of ascension of Hadoop platform.Based on the study of commonly used job scheduling algorithm on Hadoop platform and the principle of ant colony algorithm, this paper implements the purpose of the assign optimal resource to each task, in order to improve the platform’s overall performance and resource utilization. To improve the algorithm in these areas: resource scheduling, load distribution and parameter configuration. This paper proposes a dynamic adaptive job scheduling algorithm based on Hadoop platform. This algorithm monitors the change of each computing resource nodes in real time. And compute nodes through the feedback mechanism pass this information to the control node. The latter makes dynamic adjustment by improving the related parameters in the algorithm. It implements the purpose of assign the optimal resource for each of tasks.Finally, this paper sets up the Hadoop platform; it through the experiment completed analysis of run efficiency of the dynamic adaptive job scheduler. It verified the proposed scheduling algorithm can effectively improve the efficiency of job execution in a heterogeneous environment. It effectively reduces the total running time; improves the overall performance of the platform.