Research On File System Optimization And Shared File System For Non-Volatile Memory

Emerging information technologies,such as cloud computing,big data,and the internet of things,urge modern computer systems to efficiently process and store a large amount of fast-growing data and intermediate data in the rapidly developed society of digitization,networking,and intelligence.However,traditional block-oriented storage systems cannot effectively meet increasing storage requirements due to the inefficiency of block devices and multiple storage-layers.Emerging Non-volatile Memory(NVM)technologies with many advanced features,such as high density,near-DRAM speed,byte-addressability,and persistence,promise to improve the efficiency of data access by constructing NVM-based storage systems.The introduction of non-volatile memory storage devices will bring many challenges to the storage hierarchy.Traditional system software and applications need to be tailored or redesigned for exploring those new features.This thesis proposes the file system optimization and shared file system for NVM from three aspects,including multi-granular space management scheme of in-memory file systems,NUMA-aware in-memory file system,and efficient shared in-memory file systems for co-resident virtual machines.Different from previous work,this thesis fully exploits the advanced features of NVM to boost the performance of storage systems.The major contributions of this thesis include:1 Multi-granularity space management scheme for in-memory file systems.With the application of storage class memory and hardware acceleration,the write routines in in-memory file systems pose considerable overhead since repeatedly allocating free blocks and constructing file mapping structure are time-consuming.Thus,we propose a multiple granularity space management scheme to boost write performance.The scheme employs multiple granularity structured blocks whose mapping structure is proactively constructed to slash the overhead of allocating new space and constructing file mapping structure.Based on the new scheme,dedicated space allocation algorithms for different write modes are designed to exploit these structured huge blocks.2 NUMA-aware in-memory file system.The NUMA system that consists of several nodes connected by the Interlink has an asymmetric memory bandwidth and latency.Existing in-memory file systems suffer the performance penalty on NUMA machines as their layout,physical space management,and file access operations are all unaware of the asymmetric memory bandwidths and latency among NUMA nodes.Thus,we design an efficient NUMA-aware in-memory file system which distributes metadata and file data on all nodes to effectively balance loads of file requests on different nodes and develop three additional methods to optimize the file access performance,including node-oriented file creation algorithm,file-oriented thread binding algorithm,and a buffer assignment approach.3 Shared in-memory file system for co-resident virtual machines.The shared file system for the virtual machines on the same physical machine and the underlying physical machine is an effective approach for improving the performance of file transfers.Emerging non-volatile memory is promising high-performance storage for shared files.Existing shared file system designs rely on either network devices or virtual I/Os,which cannot fully exploit the benefits of NVM.Thus,we propose a new design of shared in-memory file system that supports efficient file access on the shared files of co-resident virtual machines and the underlying physical machine.The design sets up a dedicated NVM zone on the physical machine as the sharing storage for the shared in-memory file systems.The files of the shared in-memory file system are organized in a “shared page table” that is in the same form of the page table of the processes.The proposed design highly reduces the overhead of I/O software stacks and redundant data copies by directly accessing files using virtual address space and hardware MMU.Moreover,synchronous mechanisms are designed to parallelly access the shared files.We conduct extensive experiments on the proposed space management scheme,NUMA-aware file system,and shared in-memory file system with standard tools.Experimental results show that the proposed techniques can boost the performance of file access and provide high-performance data storing and processing for upper applications.This thesis optimizes the performance of in-memory file systems and design shred in-memory file systems by fully exploring the advanced features of NVM,which can properly meet the challenges of new storage systems and stimulate the development of cloud computing and big data.