The Design And Implementation Of A Storage System Simulator Based On Lustre File System

High performance computing as a frontier branch of computer science, is now in a stage of rapid development, and plays an important role in the scientific research area and technology service industry. National Super Computing Center(NSCC) in Tianjin, for example, uses TH-1A as its business host, which the scope of its application in adittion to the scientific computing, not only covers the petroleum exploration, biological medicine, cartoon rendering, but also covers the project of government informazation, city of wisdom, intelligent traffic and other fields. TH-1A has been running in NSCC for nearly 4 years. The four years running information shows that, TH-1A system design is reasonable, has a good comprehensive performance, system is stable and reliable. TH-1A has successful completed many large, typical and complex computing tasks. TH-1A uses Lustre file system as its storage system, so its computing nodes is separated from storage nodes. When facing the application which have intensive and plenty of I/O requests, Lustre showed some inadaptability, which limited the computer system to give full play to the overall performance.Because of the problems exposed above and the need of the next generation of high performance computer storage system design. We should simulate the storage system performance before the design of storage structure been determined. Simulation of the storage system is an effective way to evaluate the system performance and to obatain the optimal design scheme. At the same time, in the study of new computer system structure, especially new type of storage system structure, it's unable to evaluate the performance of the new system in the real system situation. In this situation, storage system simulator also became the best tool to explore and research the new storage system structure.In this paper, we realized a hierarchical storage system simulator base on Lustre file system. The Lustre file system is a typical parallel file system, which is widely used by the supercomputers ranks in TOP500. How to promote Lustre file system's performance, making it better adapt to the complex application access features, becomes the concern of many researchers these years.This paper mainly focuses on the following studies:(1) Study the current typical I/O intensive application requirements for storage system performance. Combined with the Lustre file system I/O process and system structure, we pointed out the problem that the Lustre file system may come across when facing I/O intensive applications.(2) Based on the research(1), we compared the main method of the current software simulators. And then put forward our design method of Lustre system simulator in this paper.(3)Implement our design of Lustre file system simulator which named Lustre Sim. And through the Lustre with TH-1A super computer system performance test comparison, verify the correctness and usability of Lustre Sim.(4) Study the key factors influencing the Lustre file system performance, and put forward the proposals on the impact of these factors on the performance evaluation. And then put forward our experiment scheme. At last we used Lustre Sim to do the experiment, points out that how these factors affect the I/O performance of the Lustre file system.