Research On Key Technologies For Persistent Memory Storage System

Storage systems are facing severe challenges in dealing with the explosive growth of data;meanwhile,new applications such as artificial intelligence also impose extremely stringent performance requirements on throughput,latency,and scalability of storage systems.Emerging byte-addressable persistent memory(PM)stores data like hard disks that survives power outages,and delivers low latency and high bandwidth that are close to DRAM.However,such new devices also drive us to rethink the new architecture of storage systems: persistent memory exhibits extremely low latency,making the software overhead of traditional storage systems increasingly obvious,and the unique hardware features also make it difficult for storage systems to fully exploit their hardware performance.To this end,we propose to redesign the persistent memory storage system architecture,and conduct research from the aspects of operating system,networking,and storage system,as follows.· In response to the problem of high software overhead and low scalability,we propose a new persistent memory file system architecture named Kuco based on kernel and userspace collaboration.Kuco extends the storage stack from kernel to userspace,and utilizes a kernel thread to manage the metadata and enforce protection.To prevent the kernel thread from becoming a bottleneck,Kuco introduces collaborative indexing,two-level locking,and versioned reads to offload time-consuming operations from kernel to userspace.Experiments show that the performance of Kuco is 16× higher than existing systems for metadata operations.· To improve the scalability of reliable connection-based RDMA,we propose an remote procedure call primitive named Scale RPC.Scale RPC divides client connections into different groups via a connection grouping mechanism,and serves each group in a time-sharing multiplexing approach,thereby avoiding cache contention in the network card.Experiments show that Scale RPC exhibits comparable scalability to unreliable connections.· To reduce the tail latency of transactional systems when processing workloads with high conflict rate,we propose a new concurrency control protocol named Plor based on pessimistic locking and optimistic reading.Plor requires a transaction to lock the data item before actually reading it,however,it allows the transaction to ignore lock conflicts without been blocked.Conflict detection is delayed to the commit phase,ensuring that transactions are committed in the timestamp order.Experiments show that the throughput of Plor is comparable to that of optimistic concurrency control protocol,while reducing the 99.9% latency by 12×.· Persistent memory updates data with a granularity of 256 bytes,despite its byteaddressability.To better utilize such devices,we propose a log-structured persistent memory key-value storage engine named Flat Store.The central idea is batching the network requests,thus merging small updates to bigger ones,which reduces the number of writes to PMs.At the same time,we introduce a pipelined batching mechanism to reduce latency while without compromising the batching opportunity.Experiments show that Flat Store improves throughput by 6.3 times compared with existing systems.