Design And Implementation Of File Layout In SAN Parallel File System

File layout which manages the mapping relationship between logical offset of file and physical address of storage device is important for a file system. Sound file layout mechanism can effectively reduce the disk seek time required to access the file. It is an important way to improve the performance of data access in parallel file system.Redbud parallel file system (RFS) is a Storage Area Network (SAN) parallel file system that the High Performance Computing Research Institue of Tsinghua University developed for 863 key project "critical mass storage technology". The writer involved in the project and was responsible for designing and implementing the file layout in RFS.Existing parallel file systems can't obtain high I/O performance of large file sequential read and write. Taking into account the characteristics of SAN parallel file system, this paper designs and implements extent-based file layout. The main work of this paper is as follows:First, this paper describes the RFS architecture, as well as the function modules at client and metadate server side respectively, and designs the RFS processes to read and write files.To address the shortage of the block-based layout used in existing parallel file system, this paper designs processes of operating extent-based layout, and the methods of dividing and merging extent, and proposes meeting the minimum requirements strategy, space continuous allocation strategy, and extents real-time merger strategy, to construct the mechanism of extent-based layout, which can reduce overhead of network communication and system, and improve I/O performance of large file sequential access.This paper introduces extent-based layout to the method of data striping storage. It designs stripe parameter setting model, and striped extent-based structure model, and proposed striped extent-based structure non-striping transforming strategy, space continuous allocation strategy, and merger based on flag match strategy, to construct the mechanism of extent-based striped layout, which can improve I/O performance of file access and system parallelism, and reduce the complexity of system implementation.Finally, this paper realizes prototype system on RFS, builds the test environments of single-client and multi-client, and uses iozone to test the throughput of the prototype system. Test results show that throughputs of the prototype system reading or writing files are much higher than those of NFS, equal or little higher than those of EXT3. That means the designed mechanism of file layout can effectively improve the I/O performance of the parallel file system. In addition, within the capacity of the device I/O, the aggregate throughputs of the prototype system reading or writing files maintain linear growth with the increase of the clients. That means the designed mechanism of file layout can make parallel file system be highly scalable and I/O performance be stable.