Scalable I / O Research And Reference Implementation

In the field of file system and I/O system, it is the research object to improve the performance and scalability. In the areas of disk storage system, network file system, parallel file system and parallel I/O interface, many research results have been gotten. But, for the I/O of scalable parallel system, there is neither unique standard nor deep understandings of performance factors. The scalable I/O (SIO) research focuses on five aspects including hardware system, operating system support, language, compilers and runtime system support, I/O characterization, and parallel applications. This thesis puts emphasis on the design of a standard scalable I/O interface, the design of a scalable file system, the implementation and evaluation of a reference implementation on the massively parallel processing system and the prototype on the system with cluster architecture.This dissertation presents a new I/O interface, called Scalable I/O Low-level Application Programming Interface (SIO-LLAPI), that provides file system implementers with a simple and performance-driven low-level interface to support high-level parallel I/O interfaces efficiently and effectively, and provides parallel I/O researchers the fundamental platform. This API proposal is a part of work of Scalable I/O Initiative of NSF of United States.In this dissertation we describes a reference implementation of SIO-LLAPI on the Intel Paragon multicomputer. The implementation provides the file system structure and file striping algorithm compatible with Intel Parallel File System (PFS), and runs either inside the kernel or as a user level library. The implementation supports the core features and some optional features of the low-level API, including the scatter-gather addressing read and write, asynchronous I/O, file control operation, client caching and prefetching mechanisms, file access hints mechanism, collective I/O and a highly efficient file copy. Our preliminary performance evaluation with the implementation shows that the SIO-LLAPI provides opportunities of significant performance improvement, is easy to implement, and helps the research of parallel I/O with API flexibility. The reference implementation is helpful for the implementers to understand the SIO-LLAPI standard, and to get the...