| In the last decade and half, High Performance Computing (HPC) has acquired continuous huge performance leaps. From 59.7 Gflops for the fastest machine in the world in 1993 on the Linpack benchmark to exceeding the petaflops barrier in 2008, the computational power available to today's applications is unprecedented. Recently, HPC community is increasingly deploying multi-core microprocessors in HPC infrastructures in order to sustain this performance growth. However, the advancement and performance potential of multicore systems comes at a cost in complexity of both the hardware and the software stack. This translates into significant challenges to programming such systems as well as to extracting high performance from them. In our research, we endeavor to investigate novel solutions to the problem of extracting high-performance.;In this dissertation, we advocate for the use of virtualization as an alternative approach to the traditional operating systems for the next generation multi-core HPC. In particular, we investigate rigorously the performance ramifications of paravirtualization for HPC benchmarks and applications. In addition, we investigate an efficient mechanism for shared-memory communication between HPC applications executing within virtual machine (VM) instances that are co-located on the same hardware platform. This system, called Vshmem, implements low latency inter-VM communication mechanism that allows the programmer to selectively share memory regions between user-space processes residing in collocated virtual machines. Our contributions addressed the two limitations of virtualization for HPC software stack and our results revealed that HPC can leverage the power of virtualization as technology trends drive multi-core architectures forward. |
