The Design And Implementation Of A New Storage System For Distributed Non-Volatile Memory

In the era of the Internet,the amount of data has exploded,which makes storage systems one of the key factors affecting the performance of computer systems.Non-Volatile Memory(NVM)has the advantages of persistence and byte addressing.The read and write performance of NVM is close to DRAM.Remote Direct Memory Access(RDMA)has the advantage of high throughput and low latency.Combining new storage and networking device is an effective choice for building high performance storage systems today.However,there will have problems in data consistency,management efficiency and life of NVM.It is difficult to directly exploit the performance advantages of new hardware by directly using the existing local file system to build a storage system.Therefore,this paper takes the new storage system for distributed non-volatile memory as the research and design goal.First,this paper introduces the development of distributed storage system architecture from two aspects of storage and network,and analyzes the inconsistency of non-volatile memory and the low efficiency of local memory management.This paper designs a new storage system architecture for distributed non-volatile memory.Second,the data consistency of NVM depends on the CPU actively executing hardware instructions through flushing cache,and such instructions are extremely expensive,which seriously affects the performance of the CPU.RDMA reads directly without the server-side CPU,so the server-side CPU do not know the write event,and cannot flush data in time.Once system crashes,the data inconsistency will occur.In view of the above problems,this paper proposes a consistency mechanism for distributed non-volatile memory file systems.The operation log is designed to maintain the crash consistency of the system by writing the type of each operation,the metadata,the address and size of the data to the log.By using RDMA immediate data,the server-side CPU actively flushes data to maintain system crash consistency while completing data transmission.This paper implements asynchronous data persistence on the server side,and uses the asynchronous thread to flush metadata,data and log of the server side to improve the processing power of the system.The prototype system is implemented on the basis of Octopus.Compared with Octopus,the consistency mechanism has a performance impact of less than 1%.Then,the overhead of the system software I/O stack will become a bottleneck in the distributed storage system after replacing the traditional network protocol with RDMA.About the above problems,this paper proposes a local NVM management mechanism that directly allocates and manages NVM.It does not rely on the local file system and reduces the I/O stack copy overhead.This paper designs a lightweight log consistency guarantee strategy,which records the type and address of each operation,and modifies the log tail pointer to ensure failure atomicity.This paper designs the index structure of hybrid memory,which stores the free list in DRAM,improves the efficiency of memory management.This paper designs a strategy combining global space and thread space.The operation in thread space does not need to be locked,which reduces the resource competition overhead and improves the scalability of the system.This paper designs a wear strategy based on double-linked list and threshold,which maintains an active linked list and a free linked list in a thread space.When the memory block reaches the threshold,it is released to the global space,which ensures that the write wear of the NVM block is effectively reduced while the memory footprint is low.The prototype system is implemented on the basis of Glibc allocator,it respectively improves performance by up to 6.4% and 101.5% for Makalu and NVMalloc under single thread.Finally,this paper implements a new storage system prototype CCMSS for distributed non-volatile memory,and uses Fio and TestDFSIO tools for testing.We observer that CCMSS respectively improves performance of write by up to 4.1 times,2.7 times and 6.7% for NVFS,GlusterFS and Crail.