A Traffic Aware Hybrid Write Buffer In High Performance Computing

While high performance computing(HPC)systems are moving toward the exascale era,the I/O performance remains one of main bottlenecks,especially for many data intensive scientific applications.Most applications running on HPC systems are write-heavy.The exemplar applications mainly perform the numerical simulations.Moreover,the write requests issued by these applications are bursty because the applications often generate and store a large amount of intermediate results.The bursty random writes to HDDs could significantly degrade the performance of data-intensive applications running on HPC systems.Meanwhile,current HPC storage systems still use hard disk drives(HDDs)as their dominant storage devices,which suffer from disk head thrashing when accessing random data.New storage devices such as solid state drives(SSDs),which can handle random data access much more efficiently,have been widely deployed as the buffer to HDDs in many production HPC systems.Burst Buffer has also been proposed to manage the SSD buffering of bursty write requests.Although Burst Buffer can improve I/O performance in many cases,we find that it has some limitations such as requiring large SSD capacity and harmonious overlapping between computation phase and data flushing stage.In this paper,we propose a scheme,called TABB(a traffic-aware SSD Burst Buffer),to improve the Burst Buffer by addressing the above limitations.We carefully analyze three common access patterns in HPC systems and develop a novel traffic-detection method to detect the randomness in the write traffic.Based on this method,only the random writes are buffered to SSD and other writes are deemed sequential and propagated to HDDs directly.In order to overcome the difficulty of perfectly overlapping the computation phase and the flushing stage,we propose a pipeline mechanism for the SSD buffer,in which the data buffering and data flushing are performed in pipeline.Finally,we propose to use the AVL tree structure to store the sequence information of the data.In the same experimental environment,compared with Burst Buffer,the results show that the proposed TABB scheme can improve the write performance by more than 50% on average by using less SSD space.