| This dissertation addresses the use of distributed memory to improve the performance of state-of-the-art virtual machines (VMs) in clusters with gigabit interconnects. Even with ever-increasing DRAM capacities, we observe a continued need to support applications that exhibit mostly memory-intensive execution patterns, like databases, webservers, scientific and grid applications. In this dissertation, we make four primary contributions. First, we fully survey the history of the solutions available for basic, transparent distributed memory support. Then, we document a bottom-up implementation and evaluation of a basic prototype whose goal is to move deeper into the kernel than previous application-level solutions. We choose a clean, transparent device interface capable of minimizing network latency and copying overheads. Second, we explore how recent work with VMs has brought back into question the memory management logic of the operating system. VM technology provides ease and transparency for imposing order on OS memory management (using techniques like full virtualization and para-virtualization). As such, we evaluate distributed memory in this context by trying to optimize our previous prototype at different places in the Xen virtualization architecture. Third, we leverage this work to explore alternative strategies for live VM migration. A key component that determines the success of migration techniques has been exactly how memory is transmitted and when. More specifically, this involves fine grained page-fault management either before a VM's CPU state is migrated (the current default) or afterwards. Thus, we design and evaluate the Post-Copy live VM migration scheme and compare it to the existing (Pre-Copy) migration scheme, realizing significant improvements. Finally, we promote the ubiquity of individual page frames as a cluster resource by integrating the use of distributed memory into the hypervisor (or virtual machine monitor). We design and implement CIVIC: a system that allows un-modified VMs to oversubscribe their DRAM size to larger than a given host's physical memory. Then, we compliment this by implementing and evaluating network paging in the hypervisor for locally resident VMs. We evaluate the performance impact of CIVIC on various application workloads and show how CIVIC allows for many possible VM extensions such as better VM consolidation, multi-host caching, and the ability to better coordinate with VM migration. |
