Towards a Flexible and Energy Adaptive Datacenter Infrastructure

The sustainability concerns of Information and Communication Technology (ICT) go well beyond energy efficient computing and require techniques for minimizing environmental impact of ICT infrastructure over its entire life-cycle. The number and popularity of large scale datacenters that host various Internet services, has increased significantly in the recent past. Electricity costs contribute to more than 31% of the overall costs in these datacenters. The increasing energy demand coupled with emerging sustainability concerns requires a re-examination of power/thermal issues in datacenters from a larger perspective of short term energy deficiencies and thermal constraints and ways to make operation of these datacenters more sustainable. The energy deficient scenarios arise for a variety of reasons including variable energy supply and inadequate power, thermal and cooling capacities. Traditionally, ICT infrastructure is overdesigned at all levels from chips to entire datacenters and ecosystem. The paradigm explored in this thesis, called energy adaptive computing or EAC is to replace overdesign with rightsizing coupled with smarter control. The goal of the Energy Adaptive Computing (EAC) paradigm is to address more directly the issue of sustainability of ICT. This is done by attempting to reduce the carbon footprint of the infrastructure via two mechanisms in addition to intelligent energy management: (a) replacing the wide-spread overdesign of the infrastructure components with rightsizing coupled with smart control to handle occasional overshoot in resource-particularly the energy-requirements, and (b) operation on renewable sources of energy. Renewable energy sources often have variable output and also require intelligent adaptation to the energy envelop.;After a brief introduction to EAC, the challenges associated with EAC in various environments in terms of the adaptation of the workload and the infrastructure to cope with energy and cooling deficiencies, are laid out in detail. This thesis focuses on the issues related to realizing EAC in a cluster environment inside a datacenter. There are three cluster-EAC scenarios studied in detail in this thesis. (1) First, this thesis presents a controller called Willow that aims at achieving energy adaptation in a datacenter environment, and addresses the problem of simultaneous energy demand and energy supply regulation at multiple levels from servers to the entire data center. The proposed control scheme adapts the assignments of tasks to servers in a way that can cope with the varying energy limitations. (2) Second, this thesis describes the design and implementation of energy adaptation mechanisms for data centers with potentially multiple tiers of service. Energy adaptation is realized by intelligent allocation of energy at various levels of the hierarchy and shutting down of over-provisioned servers. It is shown that energy adaptation could substantially reduce the power drawn from the conventional electric grid and support most datacenter operations with renewable energy sources and yet provide the required quality of service. This is achieved via coordinated control operations at different time granularities and planning strategies for executing the control operations in order to support different workloads without violating their delay bounds. (3) Finally, this thesis proposes a flexible and energy adaptive object storage framework that can adapt to variations in available or consumable power and its performance is investigated in the context of deduplicated virtual machine disks. The design and implementation of a prototype of the object storage framework is presented. The object storage framework has an adaptive replication mechanism and an adaptive consistency model for the replicas. The replicas of deduplicated virtual machine disks are managed dynamically to provide improved performance and to adapt to power constraints. Smart control techniques are proposed to cope with the power constraints either introduced as a result of increasing node density in the storage arrays or introduced when a mix of renewable (green) and conventional (brown) energy sources are used to power the datacenter. The experimental results demonstrate the ability of the framework to dynamically adapt to the changes in workload and power constraints and minimize adverse performance impacts.