Design And Implementation For Sfs-High Performance File System For Storing Small Files

For a long time, the performance of other computer components such as CPU,main memory, etc is improving quickly, while the performance of hard disk drives,due to the limit of the intrinsic mechanical movement, has little improvement,therefore becoming the bottleneck of the system performance. In some certaincircumstances, the web application environment for example, there are large quantitiesof small files (KB level) storing in web servers (10million to100million), which isstill a problem for existing file systems to handle. How to effectively manage the smallfiles in web servers in order to improve the I/O performance of small files has been aresearch hotspot both in the academic circle and industry.This thesis thoroughly analyzes the optimization of storing small files, designsand implements a file system named Sfs to adapt the application environment of smallfiles. The main work includes the following:1. Conclude the IT development trend and present the urgent demands and criticalchallenges in the current network and cloud storage age.2. Analyze the main structure and part of the implementation details, especiallythe performance optimization strategies for small files. In this thesis, not onlythe kernel supported local file systems such as ext2, ext3, xfs, reiserfs, etc. areanalyzed, but also some storage plan designed by well-known networkcompanies and open-source project groups for their service features, forexample, the Facebook’s Haystack, Taobao’s Tfs, and so on.3. Elaborate on the server file system Sfs designed for the small file applicationenvironment. To the disk layout, Sfs discards the method of dividing the diskpartitions with fixed size, and applies the strategy of dividing the stripes withdynamic chunks, to meet the storage needs of adapting the files in varioussizes. Simplify the file metadata, which not only reduces the I/O interactive times, but also saves memory to accommodate more file metadata. To the filesystem interface design, Sfs provides simple but effective API, whichimporves the processing efficiency of the file system. At the same time, Sfsdevises the small object memory pool to improve the file system processingefficiency.4. Devise the file system benchmarks, which are used in testing the file systemI/O throughput and IOPS, to demonstrate the effectiveness of Sfs whenapplied in small file application environments.The test results show that, with rational design and simplified inmplementation,Sfs has better I/O throughput performance and higher IOPS when dealing withsmall files.