Research On I/O Optimization Method Based On High Resolution Atmospheric Simulation

With the development of high-performance computing,computing power has been greatly improved.While limited to investment and complexity,I/O performance improves slowly.Therefore I/O performance has become the bottleneck of many HPC applications on today's HPC systems,and improving the performance of application's I/O is particularly important.However,I/O system has the problems of high latency,deep sharing,changeable access patterns and high cost,and the high end applications have been plagued by I/O,which is characterized by strong uncertainty and vibration.At the same time,the I/O behavior of HPC applications is mostly complex,analyzing and understanding the I/O behavior of high performance computer applications is the key to improving performance.First of all,this paper designs and implements a lightweight I/O framework that supports flexible I/O configuration,I/O adaptive tuning,but also supports synchronous and asynchronous mode.The framework is different from ADIOS,it is a lightweight framework that addresses the problem of computing and I/O performance scalability inconsistency,thus it releases the scalability of computation.Unlike PIO,the framework supports automatic I/O tuning and flexible configuration.We take GRAPES as an example to demonstrate the effectiveness of the framework on the IBM P460 system.Secondly,we design and implement an I/O trace and analysis tool for HPC applications.Our I/O trace tool can collect the I/O trace from the computing nodes and conduct low-overhead storage of the trace on-line.We take the statistics of application's IOPS(IO operations per second),I/O Bandwith,data request distribution and I/O activity of each node as metrics to quantify the I/O characteristic and issues.On the Sunway TaihuLight supercomputer,NPB-IO,MPI-IO benchmarks as well as real world applications are used to validate our I/O trace and prove the effectiveness.