Virtual clusters: Resource management on large shared-memory multiprocessors

Despite the fact that large scale shared-memory multiprocessors have been commercially available for several years, system software that fully utilizes all of their features is still not available. These machines require system software that is scalable, supports fault containment, and provides scalable resource management. Software supporting these features is currently unavailable, mostly due to the complexity and cost of making the required changes to the operating system. One proposed alternative is to partition the hardware into small units; however, hardware partitioning limits resource sharing flexibility, rendering the system unable to adapt to dynamically changing workloads.; Virtual Clusters, an alternative approach using virtual machine technology, provides all the features necessary to support large shared-memory multiprocessors at only a small fraction of the development cost of modifying the operating system. This approach effectively turns a large scale shared-memory multiprocessor into a virtual cluster that supports fault containment and heterogeneity, while avoiding operating system scalability bottlenecks. At the same time, Virtual Clusters preserve the benefits of the underlying shared-memory multiprocessor by implementing dynamic, fine-grained resource sharing, and by allowing users to overcommit resources such as processors and memory.; In this thesis, we describe the resource management aspect of the Virtual Clusters approach, which requires a scalable resource manager that makes local decisions with limited information while still providing good global performance. The resource manager must also be aware of issues related to fault containment and virtual machines in order to support fault containment and provide good performance. We describe our experience with a prototype implementation on a 32-processor SGI Origin 2000 system. We show that execution time penalties for this approach are low, typically within 10% of the best available commercial operating system for most workloads, and that it can manage the CPU and memory resources of the machine significantly better than the hardware partitioning approach.