Research On Heterogeneous System Oriented Parallel Programming

Combining with both traditional processors and co-processors, heterogeneous systems have shown advantages in many aspects. The traditional processors can supply general purpose computing power, which can be used to process scalar computing and complex controlling tasks. The co-processors such as GPUs and Intel Xeon Phi can supply very high performance with relatively lower power, which can be used to accelerate tasks with computation requirements. As a result, the heterogeneous system has become a new trend in high performance computing.However, with the introduction of co-processors and the increasing of the system scale, programming on heterogeneous systems has become more and more difficult. First of all, heterogeneous systems contain computing devices with different architectures, instruction sets and programming models, which increases the difficulty of software development. Secondly, the increasing of computing cores calls for highly parallelism of the algorithm itself.Addressing at this question, this paper studies on the parallel programming technology for heterogeneous systems. The paper has presented a new programming model, namely coordination programming model for heterogeneous systems. It divided the developers into two classes: application experts and computer experts. Based on this model we have designed a coordination programming framework CPF4 H to support corporately programming of the two kinds of developers. The paper also studies several optimization methods such as dynamically load balancing and memory accessing optimizations to address the performance bottlenecks. Finally we employ the programming framework to realize a Molecular Dynamics simulation on a petascale heterogeneous supercomputing, which validates the coordinate programming model and framework. The contributions of this paper can be concluded as:1. Proposing a new programming model, i.e. the coordination programming model, for heterogeneous systems. Based on the model, the paper has designed a coordination programming framework which containing application layer, system layer and struts layer. The programming model and programming framework have increase the efficiency of software development on heterogeneous systems.2. Designing a multi-level load balancing method. By using a static task distribution method with a dynamically load adjust method inter-node, a dynamical task distribution method intra-node, and fine-granular dynamical task distribution and multi-kernel division intra-device, the work load balancing of the full heterogeneous system has been achieved.3. Designing a memory level aware optimization method. By using a SOA based structure to increase the data locality, a SIMD extension based instructions to increase data reuse and to address cache miss, and a shared memory based atomic operations, the memory bandwidth bottleneck has been largely appeased.4. To validate the coordination programming model and framework, the paper employs CPF4 H framework to develop a Molecular Dynamics simulation SWP on a petascale heterogeneous system Tianhe-1A. The evaluation result shows that the coordination programming framework can improve the programming ability of the heterogeneous system, and the simulation can explore the performance of the system. The SWP simulation on 3000 hybrid nodes shows advantages to 6000 CPU-only nodes, and runs 1.4 and 1.25 times faster on 7000 nodes comparing to CPU-only and GPU-only nodes, respectively.