Flexible and efficient resource management in a virtual cluster environment

Virtual machine (VM) use in a cluster environment imposes many challenges upon a cluster administrator. As the number of VMs across a site grows, manually tracking transient site state information such as resource availability and VM locations and status while enforcing policies for running large numbers of VMs across a cluster becomes increasingly difficult. In this dissertation I focus on VM use in clusters and consider the management and efficiency problems that arise in this unique environment. I present Usher, a virtual machine management system designed to impose few constraints upon the computing environment under its management. Usher enables administrators to choose how their virtual machine environment will be configured and the policies under which it will be managed. Usher cluster administrators can push basic virtual cluster management tasks such as VM start and stop out to the users (virtual cluster creators) themselves, reducing administrator workload and allowing users to create virtual clusters on demand. The modular design of Usher allows for alternate implementations for authentication, authorization, infrastructure handling, logging, and virtual machine scheduling. The design philosophy of Usher is to provide an interface whereby users and administrators can request virtual machine operations while delegating administrative tasks and policy enforcement for these requests to modular plugins. I present an Usher scheduling plugin designed to map virtual machines onto physical machines such that an arbitrarily defined utility is optimized. I discuss possible cluster scheduling goals and present a representative cluster scheduling problem called Fair Maximum Utilization (FMU). Exploration of scheduling heuristics of varying levels of sophistication applied to FMU suggest that those which make better VM resource demand predictions and only slight schedule adjustments work well in general.