Improving MPI Communication Performance In Overcommitted Virtual Machines

After resurgence, software based machine virtualizatin technology becomes an important research topic. And using such technologies to build environments for solving high performance computing (HPC for short) tasks becomes an important trend nowadays. The introduction of virtualization layer provides many advantages that can not be provided by traditional computing model, such as resource consolidation, power reduction, improving application portability, and to help debug complex application codes.In the traditional computing model, the application processes or threads are scheduled to run on physical machines. However, in the virtualized computing model, for the introduction of the virtualiation layer, the schduling mechanisms is changed to two-layer scheduling : one resides in the guest operating system, which schedules the application processes or threads, while the other belongs to the hypervisor that schedules the virtual processors. Moreover, in current architecture, these two scheduling mechanisms work independently without any coordinations. This will adversely affects the HPC workloads including that need to communicate with each other by using MPI communication primitives during the process of computation workloads. And such negative effect becomes rather salient when the VCPUs are overcommitted (more than one VCPUs need to be scheduled on one PCPU).The research analyzed the actual reason behind the performance penalty, that is the busy-waiting mechanism of MPI library which waste a lot of CPU resource, then though analyzing the source code of MPI library carefully, the hidden reason of performance penalty is founded: two scheduling mechanisms working independently, at last,a solution is proposed to improve the communication performance when running MPI application in overcommitted virtual processors by revising the message receiving mechanism of MPI. The revised message receiving mechanism of MPI made the two scheduling mechanisms work cooperatively by reporting some scheduling information of the guest operating system to hypervisor to guarantee the scheduling decisions of hypervisor more accurate and to improve the communication performance.The results of experiments showed that the solution can greatly improve the performance of HPC applications which need to communicate with each other by using MPI primitives in overcommitted virtual machines.