Research On Energy-Efficient Improvement Methods For Storage And Computing Layer Under Cloud Computing Environment

The excess resource supply,redundant design and load-balancing algorithms are used in cloud computing data centers to guarantee the QoS and system reliability,which exposed the high energy consumption problem.However,with the expanding of the system scale,the problem is getting worse.Building the energy efficient cloud computing system is an important issue currently for the information industry,especially,solving the optimization problem of cloud storage and computation model is the key to improve energy efficiency in the cloud computing center.In this dissertation,we did a series of studies on energy issues of cloud storage and MapReduce computation model,aiming at improving the energy efficiency of the whole cloud computing data center by energy-efficient storage layer and computing layer in cloud computing data center.The main contributions of this dissertation are as follows:(1)The current research situations of the energy-efficient cloud storage and MapReduce computation model are summarized.First of all,we discussed the energy consumption optimization problem of the cloud storage system from two aspects:the energy-saving method based on hardware and the energy-saving method based on scheduling.And we divided the energy-saving method based on scheduling into three categories:the method with node scheduling,the method with data scheduling and the last with cache pre-fetching technology,as well as we made a comprehensive comparison among them.Secondly,we summarized the existing research results of energy-efficient MapReduce computation model.(2)Energy-efficient algorithms for the distributed storage system based on data storage structure recon figuration.We solved the problem of complete coverage for data availability by constructing metric matrix for data-block availability.And we built the energy-saving model for distributed storage system in which the RACK is divided into two distinct storage area:Active-Zone and Sleep-Zone.According to the activity factors,we configured data storage area,and according to data center load rules,we timely hibernated the servers in Sleep-Zone to save energy.(3)Energy-efficient algorithms for distributed storage system based on the symmetric data block storage strategy.We proposed symmetric data block storage strategy and energy-efficient algorithms for this strategy,and implemented task transfer and replacement between DataNode nodes with the same storage structure without affecting the data reliability,high efficiency and scalability of the original HDFS.Then by sleeping idle nodes produced from task scheduling for energy-saving purpose.(4)Adaptive metadata models and algorithms for energy-efficient cloud storage system.After making a lot of studies on the existing cloud storage architectural pattern,storage model and strategy,Metadata management and organization and QoS constraint guarantee,we proposed adaptive metadata dynamic modeling and management method for energy-efficient cloud storage system.Furthermore,we designed adaptive disk-level storage structure,data block write&read strategies and energy-saving modes switching algorithm around the energy-saving metadata model,which effectively solved the matching problem between system and energy-saving algorithms.(5)Research on energy modeling and optimization analysis for MapReduce.In order to improve the MapReduce framework utilization of energy consumption,we build the energy consumption model for the MapReduce framework.First,we modeled the MapReduce task-level energy consumption,and then proposed task-level energy consumption models based on CPU utilization estimation,energy accumulation of major components and average energy consumption estimation respectively.Furthermore,based on the three task-level energy consumption models,we built the MapReduce job energy consumption model.Second,after analyzing energy consumption optimization problems based on energy consumption models,we discussed how to optimize MapReduce from three aspects;optimizing energy consumption of MapReduce job execution,reducing energy consumption of MapReduce task waiting,and improving energy efficiency of the whole MapReduce cluster respectively.At last,we proposed data placement strategy under heterogeneous environment to reduce energy consumption of MapReduce task waiting and minimum resource(slot)allocation strategy with the deadline constraint to improve energy efficiency of MapReduce job.(6)The temperature-aware energy-efficient task scheduling strategies for MapReduce.By studying the existing MapReduce task scheduling models,we found that the scheduling system does not care the current temperature condition of the nodes with idle slots.Then we added CPU temperature conditions to the decision information of task scheduling,in order to protect the overall job progress from being affected by a few task execution nodes with high temperature.Finally we proposed two solutions to implement the algorithms:one is based on heartbeat information modification and another is based on health monitoring scripts.