Research And Implementation Of NVM-based High Performance User-space File Systems

During the age of information,the computer data accumulated year by year has an exponential growth.With human gradually rely on information,big data processing has become one of the important requirements for information processing system.Big data processing puts forward higher demand for storage system,mainly focusing on bandwidth,latency and reliability.For a long time,due to the price of memory,capacity,durability and other limitations,the large capacity of persistent data often need to be stored to some mediums like disk,which formed the hierarchical storage system which from cache,main memory to disk.Due to the limited performance of persistent storage such as disk,various optimization techniques based on locality attempt to fill the gap between the storage and memory.But with the rapid development of new storage,such as nonvolatile memory,the performance of persistent storage is greatly ascend.The combination of Memory and the advantages of the storage is expected to improve the overall performance of the storage system.On the other hand,new storage devices also bring new challenges to the design of storage systems.In order to fit the persistent memory and make full use of its characteristics of high capacity,low latency and durability,the traditional system software needs to be adjusted accordingly.In terms of big data processing and data intensive applications,this paper based on the persistence of memory storage system for research,mainly focus on the design and optimization on the memory management system and file system of the operating system,improving the efficiency of data processing.This paper focuses on the system software level,including storage management,file system,process management and system reliability.The main work and innovation of this paper are as follows:(1)This paper proposed a file indexing mechanism for persistent memory.The organization of a large number of files in traditional file system is usually optimized by the directory file.The traditional way of indexing and optimization,such as B tree and hash table,faces the tradeoff of the batch access and random access,or can not be applied to the persistence of memory because of the disk optimization.More importantly,hierarchical organization based on directory files requires the file system to do hierarchical parsing,which brings the software overhead to be reckoned with.Aiming at the shortage of traditional indexing methods,this paper try to organize and index file by page table for the persistence of memory.And this paper through the full path name parsing and leverage MMU hardware to accelerate the file locating,so as to reduce the file access latency.Experiments show that the improved file system with this mechanism has 66% to 6.7 times performance improvement in terms of latency.(2)A storage management scheme orienting persistent memory was proposed.Persistent memory will co-exist with traditional memory for a long term due to its shortcomings of persistent memory in terms of performance and life expectancy.In this memory structure,it will be a main problem for system software designers that how they identify and manage persistent memory.This article used the memory management system in Linux operating system to manage persistent memory.First,it allocates a fixed virtual address space for the persistent memory.This space is reserved from the user space in the 64-bit system address space.Thus users can access the file just by reading the kernel page table and modifying corresponding permissions.And they do not need to do pageby-page mapping so it will improve access performance and facilitate file sharing and subsequent user-directed design.In order to realize this scheme,the mechanisms such as process initialization and page fault exception handling need to be adjusted correspondingly.In addition,a new memory zone used for managing persistent memory was designed,which will allocate and statistic persistent memory separately.(3)Design and implement a model for migrating traditional file systems to persistent memory.Based on the new file indexing mechanism,this paper designs a model for porting traditional file systems to persistent memory.That is,it uses the new indexing mechanism to replace the directory mechanism in the traditional file system and improves the parsing and cache process in the kernel virtual file system.On the basis of the model,this article builds a kernel library – libptree.And experiments show that the traditional file system can be easily ported through theis library.For example,the migration cost of the ext2 system is only about 200 lines of kernel code.