Fortran Code Reconstruction Based On Mapreduce Programming Model

Fortran language has been widely used to solve computationally intensive problems in science and engineering.Due to `the emergence of multi-core architecture,the pursuit of Fortran parallelism has become an important goal in the field of scientific computing.However,in the era of big data,the parallel Fortran program of multi-core architecture still can’t adapt to the explosive growth of data due to insufficient computing resources and poor scalability.As a business computing model,cloud computing not only has abundant cluster computing resources,but also has ideal flexibility and scalability,which provides a feasible solution for solving the related research problems such as insufficient computing resources in the field of multi-core parallelism.Map Reduce is an effective programming model that supports cloud computing to process massive data in parallel,and Spark platform is an efficient implementation of Map Reduce.It is a meaningful task to automatically map parallelizable Fortran code to Spark platform,At the same time,considering the complexity of cloud programming in the mapping process,Therefore,this thesis proposes an automatic refactoring and offloading scheme of Fortran code for Spark cluster.The work of this thesis mainly includes the following parts:(1)Propose a scheme of refactoring Fortran code into Spark-Map Reduce code.Firstly,the source code is converted into intermediate code to collect the semantic information of the program.Then,based on the parallel mode of Map Reduce programming model,a parallel refactoring method is proposed to realize the refactoring from intermediate code to Spark-Map Reduce code.The target code of refactoring includes two parts.One is to propose a refactoring method to refactor the parallel loop into map function and reduce function.The other is to define the Spark job application template and point out the parts that need to be filled in the refactoring process,so as to generate the matching Spark job application for the parallel loop computing task.(2)Propose a scheme of offloading computing tasks to Spark cluster.Design an offloading function library that can directly interact with Spark cluster,and use the Open MP offloading model to automatically generate calls to the offloading function library to realize offload computing tasks.In this scheme,distributed data partitioning is also achieved by extending the use of Open MP instructions,reducing the overhead of moving data within Spark.(3)The refactoring and offloading scheme proposed in this thesis is implemented in LLVM compiler system,and the visual front-end is developed.In the experiment,Poly Bench/Fortran benchmark test set is used to test the feasibility of the scheme,and the performance of the scheme is evaluated by comparing with the parallel technology of multithreaded Open MP.The experimental results show that the proposed scheme can effectively map Fortran code running on local computer to Spark cluster automatically,and the implementation of this scheme in LLVM compiler system can improve the execution efficiency of the original business.