The Design And Implement Of A Improvement For Small File Performance In Distributed File System

Distributed File System has been widely applied in High Performance Computation fields such as militaris tstrategist, weather forecast. These fields'speciality needs the distributed file system being of highly throughput and availability. The distributed file system has been the main storage architrave of HPC environment.It has two characteristics: one is the bigger capability and higher aggregation I/O bandwidth by distributed data storage; the other is abundant of the I/O access mode, such as the object-based data pattern by use object-based storage architecture.Lustre is a typical object-based distributed file system. It origins from the coda roject in Carnegie university, and has become one of the widely used distributed file system in HPC filed.It has ffavourable big file I/O performance, which has almost Peta degree aggregation I/O bandwidth. Therefore, ,because of its distributed storage architecture which placing file data and meta data separately.ite small file I/O performance is not so well,even lower than that in the local file system. In this paper, a cache-based improvement for small I/O performance, called Filter Cache is designed and implemented, which is based on the storage architecture and implemention of Lustre1.6.4,and expands the data path of the Object Storage Target.Small file I/O performance testing on Filter Cache are carried out after we finished the design and implemention, which show that Filter Cache can indeed improve small file I/O performance, read performance has been enhanced by a factor of 65% when all read I/O request hits cache.The cache hit ratio and access latency are both important for cache. In this thesis, some optimize measure are done after we study that in traditional file system cache. Our optimize method focus on the cache structure,cache hit ratio,and I/O flow.We will do more optimize work on Filter Cache in future.In the end, we do some work over on another Collaborative Object Cache structure used in distributed file system, introducing its impletion and characteristic detailedly. We proposed two important improvement for Filter Cache based on its merit.