Porting And Optimization Of OpenFOAM On The Sunway Taihulight Supercomputer

The Sunway TaihuLight supercomputer based on the homegrown SW26010 many-core processors claims the top place in the latest TOP500 list with a Linpack performance of 93 PFlops.OpenFOAM is one of the most popular open source Computational Fluid Dynamics(CFD)software which is written in C++ and not fully compatible with compilers on the heterogeneous many-core processor SW26010.We ported OpenFOAM based on SW26010's MPE(management processing element)/CPE(computing processing element)cluster architecture.To overcome the compilation incompatibility problem,we adopted the mixed-language application design.Different sparse matrix storage formats and preconditioners were applied to improve the parallelism efficiency on CPE.We also applied several SW26010's feature-specific optimizations on the hotspot of OpenFOAM to deliver a high performance,such as register communication,vectorization,and dynamic task partitioning.The experiments on SW26010 using real datasets show that the single-CG(core group)code runs about 10 x faster than the code optimized with compiler options,and the performance of single-CG is comparable to the serial implementation of Intel(R)Xeon(R)CPU E5-2695 v3.We also extended the implementation of single-core group to the large-scale cluster and achieved better scalability.The following are our original contributions:(1)We ported OpenFOAM on the SW26010 processor and adopted the mixed-language application design to solve the compilation incompatibility problem;(2)We applied different sparse matrix storage formats and preconditioners,then extended the implementation of single-core group to the large-scale cluster with MPI and Athread;(3)We applied several SW26010's feature-specific optimizations on the hotspot of OpenFOAM to deliver a high performance,such as the register vectorization,communication,and dynamic task partitioning.