Research On Key Technologies Of Small-scale High Performance Virtual Clusters

Applying virtualization technology in high performance computing is a hot topic of current research, but for small-scale hign performance computing, because of their heterogeneous hardware resources and the pursuit of peak computing speed and other characteristics, so it has own specific demand for virtualization technology, not all virtualization technologies are suitable for it. At present, researches at home and abroad focus on the whole solutions based on virtualization technology for high performance computing. However, the current research on virtualization core technology is lacking for small-scale high performance computing.To solve the above problems, this thesis in-depth studies the virtualization technology and evaluates the performance of Xen and KVM for hign performance computing through benchmarking. This experiments show KVM is more suitable for constructing small-scale high performance virtual clusters than Xen. Followed by the improvement and optimization for KVM scheduling strategy, we aim at improving performance. Finally, we put forward a solution for small-scale high performance vitual clusters. The main contents of this paper are as follows:· Firstly, we study the principles of virtualization technology, including full virtualization and para virtualization. And evaluate the performance of the popular open source virtualization technologies, such as KVM and Xen, including overall system performance evaluation and scalability for HPC systems in order to select the most suitable virtualization technology for high performance computing;· Followed by in depth analysing KVM scheduling strategy, and in order to minimize the performance overhead of virtualization, we optimize KVM scheduling strategy. Finally, through HPC Challenge (HPCC) Benchmark Suite for the performance comparison of improved KVM scheduling strategy, experiments show that optimized KVM scheduling strategy can exhibit the better performance;·Through an idea of cluster and virtual cluster, the advantages of virtual cluster are shown. Meanwhile, we introduce PXE network boot protocol, Kickstart automated installation tools, Ganglia monitoring system and Sun Grid Engine(SGE) cluster management system in Rocks cluster system.· Finally, we build a Rocks virtual cluster for high performance computing based on above two key technologies. Through Linpack micro benchmark and NPB macro benchmarks we assess Rocks virtual cluster performance, including unimproved KVM scheduling and improved KVM scheduling, experiments show that the improved KVM scheduling strategy make Rocks virtual cluster hava a better performance. Its performance is closer to the performance of physical cluster. So, it is a feasible solution for the implementation of hign performance services.This article reflects the author’s main research characteristics including the following:· This thesis studies nature of virtualization technology and evaluates the performance of Xen and KVM virtualization technology based on HPC loads, and gets the result that KVM technology is most suitable for high performance computing;· Studying the scheduling strategy of KVM virtual machine manager. Through benchmarks, the improved KVM scheduling strategy shows the better performance.· Based on above two key technologies, this thesis describes the design and construction process of Rocks virtual cluster;· By benchmarking, Rocks Clusters is an effective virtual cluster to implement the high performance computing loads;· Based on optimizing KVM CPU scheduling strategy, through benchmarks experiment shows that Rocks virtual cluster has a equivalent performance to high performance physical cluster, and so controlling the virtualization performance loss to a minimum.