The Characteristics Analysis And Performance Optimization Of Memory And I/O For High Performance Computer

With the rapid development of heterogeneous architecture and high-performance CPU technology,the peak performance of the super computer represented by the "Tianhe-2" has reached 54.9PFlops,which is developing to Eflops scale.CPU,memory,interconnection network and external devices are the main components of the supercomputer,the first three are photoelectric type,and the storage device as one of the main external devices is a combine of mechanical,electrical,magnetic,which has a large gap in the development speed.The bandwidth of accessing to memory and communication is about n GB per second,the delay is about n ns,but the bandwidth of the disk storage device is about tens to hundreds of MB per second.The serious imbalance between calculation,accessing memory,communication performance and I/O performance has greatly affected the performance of the application.As a result,we need analyze computing,memory and I/O resources whice are used by high-performance computing applications,in order to understand the operating characteristics of the application and the performance problems of the system in the memory and I/O aspects,researching the solving methods of solving problem.The paper carried out the following work focusing on the characteristics of high-performance computing applications analysis:(1)Analysing the composition structure and the current development status of supercomputers,studying the supercomputer storage systems and high-performance computing applications demand for memory and I/O.(2)Analysing the usage of memory of the high-performance computing applications on TH-1A,finding the problem of memory wasted caused by the insufficient use of memory.Studying the meaning and technical methods to optimize memory structure.Presenting an memory optimizing technique,which is based on NBD(Network Block Device)mechanism constructed using remote node memory to build page swap area.The technology uses the characteristics that the performance of accessing a supercomputer memory via network is far superior than the performance of accessing local disk,using a remote memory swap disk technology,you can swap pages to the physical memory of the remote node when the memory of local node is tight,which can effectively improve memory usage on the TH-1A.(3)Researching the technical methods of I/O characteristics analysis on high-performance computing applications.Designing a I/O Trace collection and analysis tools,which can be run efficiently on TH-1A.(4)Using this tool to monitor the I/O behavior of MADbench2 and b_eff_io which are benchmark applications,then analysing the I/O characteristics of oil seismic exploration gather trace program which is the typical data-intensive applications on the TH-1A.Understanding the I/O characteristics of these programs,which can help us targeted optimize applications and ease the system I/O bottlenecks.