Automatic Generation Of Benchmarks For MPI Programs Based On Event Tracing

Benchmarking is a crucial technique for performance analysis and evaluation in high-performance computing(HPC).However,the development of artificial benchmarks requires extensive expert manpower and often lags behind the development of HPC systems and applications.Therefore,automated benchmark generation methods have been proposed as a powerful supplement to manual benchmark generation.As the most widely used parallel programming standard in HPC,MPI is a major research focus for many researchers.Currently,the event tracing-based benchmark generation method is the mainstream for generating MPI program benchmarks in the field of HPC.Event tracing is a fundamental method for analyzing performance and understanding the behavior of MPI programs.It can obtain logical information and communication skeleton of the program during execution,which can be used to automatically generate benchmarks through the extracted information.However,with the increasing parallel scale,the trace of MPI programs requires more storage space,making it challenging to extract information from huge traces.This is exacerbated by irregular parallel programs.In addition,the benchmarks generated solely based on communication information have limited functionality,and can only restore the communication behavior of the program,while ignoring its computation performance.To address these problems,this thesis presents a series of works on the automatic generation of benchmarks based on event tracing,focusing on two main aspects:(1)Automatic generation of structured benchmarks based on grammarThis work proposes a new benchmark generation framework,Siesta,which incorporates the collection and extraction of computational information into trace compression and automatic benchmark generation.Performance metrics of the computation part between communication functions during the execution of MPI programs are recorded using hardware performance counters.The performance metric information is then extracted using a clustering algorithm,and the computation part of the benchmark is generated by synthesizing calculation proxy programs.For MPI communication functions,this work implements a compression and extraction method based on context-free grammar,which achieves efficient compression.Experimental results show that compared to existing benchmark generation methods that only record communication information,Siesta can achieve more accurate time prediction and better cross-platform and MPI implementations robustness.(2)Automatic generation of unstructured benchmarks based on sequence predictionThis work proposes an unstructured benchmark generation method,LCR,to address the difficulties of compressing irregular parallel MPI programs.Unlike structured compression methods that explicitly search for repetitive structures in the program,LCR models the trace as a time series and trains a lightweight recurrent neural network to extract information from the trace.A simulator is then used to replay the trace and validate the benchmark’s validity.Experimental results show that compared to existing structured compression methods,LCR can further reduce the size of irregular traces by up to three orders of magnitude,and accurately reproduce the performance metrics and communication patterns of the original MPI program.Overall,this work aims to tackle the challenges of generating benchmarks for performance analysis and evaluation in HPC systems and applications.The proposed methods show improvements over existing methods in terms of time prediction accuracy and robustness,as well as achieving high compression ratios for traces.