Research On OpenMP Towards Cluster Systems

Clusters of workstations or PCs utilizing commercial off-the-shelf components have emerged as a cost-effective and scalable alternative to high-end compute platforms. They are effectively used for scientific and engineering applications and are rapidly becoming mainstream. Traditionally, a message-passing programming model is matched to cluster architecture. But a shared-space programming model for clustered systems, even automatic parallelization, is still attractive for parallel computing. The OpenMP Application Programming Interface (API) is a de facto standard for parallel programming on shared memory multiprocessors. It is much easer to program and facilitates an incremental approach to the parallelization of sequential programs. Scalability and programmability are two metrics of performance of a parallel system. Combining the good scalability of clusters with the fine programmability and portability of OpenMP, the implementation of OpenMP towards cluster systems will greatly promote the development of parallel applications.This dissertation proposes an integration of the compile-time and the run-time techniques to design a general infrastructure of OpenMP on cluster. In the combined system, the run-time library builds a shared-memory programming abstract on message passing hardware platform, while the compiler generates a shared memory program augmented with run-time calls that describe the program behavior. The implementation of infrastructure in this dissertation is called OpenMP/JIAJIA. This prototype includes the source-to-source OMP2JIA compiler and the fork-join style OMP JIAJIA run-time library. Like a cc-NUMA machine, it can support the direct execution of OpenMP applications on cluster systems.In this dissertation, to improve the performance of OpenMP on cluster systems is deeply studied by data distribution, loop scheduling, and system optimization respectively. By analyzing the gap between UMA architecture and NUMA-like cluster architecture, the dominative factor of OpenMP programs on cluster is focused on, namely, whether data layout is matched to data access or not. A flexible data distribution strategy, and two effective static and dynamic loop scheduling algorithms are proposed at OpenMP language extension in order to attain good performance. It is convenient for programmer to get owner-computation by inserting adaptive extension directives. Additionally, many kinds of optimization experience and skill around three levels (programmer, complier, and run-time library) are also proposed. For example, write-vector technique, home migration, and data prefetching in JIAJIA system are all supported at language level. Reduction directive optimization is proposed by replacing lock mechanism with message passing primitive at run-time level.It is necessary for objective evaluation on parallel processing system to collect a wide...