Research And Implementation Of Distributed Storage Energy Saving Technologies Based On Ceph

Distributed storage systems are widely used in data centers, because of their conspicuous performance and reliability. However, the energy consumption of distributed storage systems is considerable and occupies significant part of the whole data centers' energy costs. Therefore, energy saving technologies of distributed storage system are very import for reducing the costs of data centers and green computing, and they have already become hot research points, both in academia and industry.Ceph, invented a few yeas ago, is one of the most popular distributed storage systems nowadays. Ceph has many advantages, like high scalability, high performance, high reliability and the petabyte scale storage space. As a result, Ceph has gained increasing attention. Yet Ceph also faces the problem of reducing power consumption. For one thing, in Ceph, data blocks and their replicas are distributed to data nodes randomly, which restricts power saving proportion of the system, and when the system scales up, the power to be saved would leave munch to be desired. For another, there is no power managing mechanism in Ceph, all nodes keep running all the time, and this will cause unnecessary power consumption when system is in low load. Considered these issues, we do the following work:Firstly, we introduce an algorithm, named PEPGO, to optimize the data layout produced by the CRUSH algorithm. Our algorithm analyses Crushmap and placement rules, which are used to describe the cluster's hierarchical relationship, and the replica placement strategy, to find the appropriate failure domains, and then split the failure domains into power groups. The data blocks' replicas will be placed in different power groups as well as in different failure domains. As a result, more nodes can be shut down, and all data sets still keep accessible. This will cause considerable energy saving when the number of nodes goes up.Secondly, we introduce a multi-level power manage strategy, which makes system in different power level according to the system's load, based on the Gear Shifting mode. The original purposes of Ceph are high performance and high scalability, but in real workload, there are many low load periods, when some nodes can be powered off for energy efficiency. The strategy we proposed can determine the system?s power state according to the system load. As a result, the needs of system?s load are met, as well as the power consumption of system can be reduced.Finally, we design and implement a power manage system for Ceph, based on the two former points. This system uses the existing Ceph's manage, configure interfaces to optimize the data layout, and traces the system load status by Ceph?s monitor interfaces. Besides, in this system, power manage mechanism of hardware and the network based waking up technology are used to perform the manage activities. The results of evaluation illustrate the system can reduce the power consumption of Ceph effectively, meanwhile, the quality of service is preserved.