Research On Containerization Scheme For HPC Applications In Heterogeneous Systems

In recent years,hardware devices with increasingly rich functions have made the computing environment more and more multifaceted.With the emergence of many accelerators,heterogeneous systems quickly seize the general computing market with their unique performance advantages,and gradually become the mainstream platform in the field of High Performance Computing(HPC).However,in the heterogeneous system,there are many factors that restrict the efficiency of programs,and during the process of application migration,most implementations lack in-depth analysis and optimization of performance bottleneck,the actual acceleration effect is far from the system performance,thus resulting in the waste of computing resources.Besides,the mutual interference of specific dependent libraries between HPC applications is also one of the reasons for the low utilization rate of system resources.Although Docker container provides a new solution for running different HPC applications in the same heterogeneous platform,due to its own privilege escalation issue and resource management mechanism conflict with HPC application scenarios,thus decreasing scalability and functionality greatly.In order to solve these problems,this article takes High Performance Linpack(HPL),a mainstream HPC application as an example.By optimizing the efficiency of program and studying the application containerization scheme,the system resource utilization and application portability are effectively improved.The article mainly includes the following three parts:(1)Combine heterogeneous systems and computing characteristics of parallel applications,analyze the data layout among multiple processes,and compare the transmission overhead of two different task load models,CPU-centric and GPU-centric.Meanwhile,based on experiments,it is verified that the latter can effectively reduce data transmission and help to accelerate the decomposition calculation of higher-order matrices;(2)Propose a performance optimization scheme for heterogeneous systems,and accelerate calculation through asynchronous iteration,introduce many methods such as double buffering,multi-stream execution,adaptive task partitioning and GPU-based multi-thread parallelism to improve the efficiency of floating-point operations of program,and the improvement effect is verified through the performance comparison before and after optimization;(3)Combine Singularity container to design a containerization scheme for HPC applications,and based on this scheme,the performance of program is compared in different running environment of single-node and multi-node respectively,thus proving the effectiveness and feasibility of the research scheme.