Research On MapReduce Fair Scheduling Algorithm In Heterogeneous Cloud Computing Environment

As an emerging technology and a business usage pattern, cloud computing is designed to provide users with safe, fast and convenient data storage and network computing services. Meanwhile, its features such as dynamic expansion and on-demand pay enable cloud computing control service costs better and reduce energy consumption, thus having broad market prospects. Since MapReduce is the most widely used distributed computing framework, numbers of organizations and individuals are committed to the improvement of its scheduling algorithms, hoping to improve the efficiency and fairness of MapReduce framework.Among the existing MapReduce scheduling algorithm, the fair scheduling algorithms are based on homogeneous environment with ignoring the heterogeneous environment, thus it is difficult to guarantee the scheduling allocate the jobs fairly. At the same time in heterogeneous cloud environment, because of the variety of resource requirements, it's difficult to guarantee the jobs' performance highly due to the same physical resource preemption. But the current research work in view of the work can be placed only consider the problem of data locality, does not take into account the node other operations affect the performance for the new job, and it is difficult to guarantee the scheduled job execution efficiency.Therefore, based on the study of MapReduce fair scheduling algorithm and jobs can be placed and other related work, aiming at the problem of how to realize the fairness of scheduling MapReduce jobs in heterogeneous in cloud environment, and puts forward the method of MapReduce fair scheduling in a heterogeneous environment in the clouds. The method establishes a MapReduce job fair evaluation model, this model through job relative execution fairness efficiency evaluation MapReduce job and resource scheduling, which can reflect the relative fair resource scheduling in heterogeneous cloud environment. On this basis, according to the MapReduce heterogeneous cloud environment fair scheduling problems, propose a fair scheduling algorithm. The jobdifict is based on resource scheduling fairness and the jobs'completeness, the jobdifict ranking of job scheduling, to ensure the fairness of scheduling MapReduce jobs in heterogeneous in cloud environment. Aiming at the MapReduce heterogeneous cloud environment in the operation can be placed evaluation problem, establish a measure of operating performance of mutual interference model, and model parameters are given for solving the multiple linear regression method and the performance prediction algorithm based on mutual interference, puts forward a MapReduce job scheduling algorithm can be placed delay measurement based on, can avoid the performance degradation due to the work of resource preemption. Finally, experiments verify the effectiveness and fairness algorithm of MapReduce fair scheduling heterogeneous in cloud environment, and validate the delay scheduling algorithm can be placed on the job.