Research On Architecture And Strategy Of Parallel File System Based On SSD

With the development of the times, in weather forecast, astronomical calculation, mineral oil and gas exploration, biopolymer calculation and aerospace military simulation and many other areas, scientific computing puts forward higher and higher demand for the timeliness and accuracy of the calculation results. Scientific computing applications change from computing intensive to data intensive and have become increasingly demanding for high I/O performance of storage system. Therefore, research on the I/O performance optimization of distributed parallel file system deployed in storage system is of great significance to improve the operational efficiency of scientific computing applications.Most of distributed parallel file systems deployed in high performance computing use excellent architecture design and are very suitable for large file sequential access, such as Lustre. They have excellent performances in the application of large files. However, these designs are not conducive to large-scale random access and small file I/O, especially in the massive small files application scenarios and the I/O performance is greatly reduced. In order to solve the serious performance degradation problem caused by large scale random access and small file access, Smart system consists of I/O middleware, scheduler module, file pattern recognition module and database. It combines solid state disks(SSD) with hard disk drives(HDD) to form a hybrid storage system, determines the actual storage location according to the file distribution rules and file pattern recognition information. By this way, Smart system solves the problem caused by large scale random access and small file access.Running standard file system benchmark on Smart system and comparing with the original Lustre file system, experiments show that, our system achieves up to 25%-89% I/O bandwidth improvement.