Research On Low Power Scheduling Technology For Heterogeneous Cluster Based On MapReduce

In recent years,with the development of Internet technology,the quantity of users and amount of data increase sharply.Processing and analyzing the huge amount of data is proved to be an urgent task.MapReduce heterogeneous cluster becomes one of most widely used and efficient data processing platform.However,unreasonable allocation of task results in low energy utilization.In this paper,we devote to researching low power scheduling problem of heterogeneous cluster based on MapReduce.Our work is as follow:As necessary basic knowledge,firstly,we conclude the state-of-art of MapReduce research.Then,we elaborate the computing model of distributed system,parallel computing system and heterogeneous computing system.Moreover,we focus on describing MapReduce computing model and execution process of Hadoop.According to differential execution situation of heterogeneous cluster,we build a scheduling model of allocating tasks to servers with different computing capacity so that the total execution power minimized,within a given time limitation.CPLEX 12.4 is applied to find solution to the combinatorial optimization problem.Based on the solution above,CloudSim3.0 is utilized to simulate distributed cluster environment and judge the performance.The performance of TeraSort and K-means clustering are employed as metric.Compared with FIFO(First In First Out)scheduler and SLO(Service Level Objective)scheduler,the experiments show that the average executing consumption of our strategy reduce by 19% and 10.5%.The results also reveal the relationship between time limitation and energy consumption.