Design And Implementation Of Shared Log In Distributed System Based On NVM

Shared logs are one of the basic building blocks of distributed systems.The increased popularity of shared logs in the last decade is due to their offered simplicity and proper-ties like Strong Consistency.Shared logs sort events received from multiple clients and replicate them across multiple servers so that the data is synchronized while the client can access it in parallel.Since it is a core building block,shared logs need to be as high performing as possible.In recent years,with the increasing scale of data in the Internet,people put forward higher requirements on the performance of data center storage system.At the same time,the development of computer network and non-volatile memory brings new opportunities for the design of shared logs.Compared with the traditional shared logs built on SSD,the shared logs built on top of NVM can make good use of its low latency and non-volatile characteristics to improve the overall performance of the system.However,due to the current design of shared logs does not guarantee persistent writing to NVM,and the overall design of the system is not flexible enough,there are still some problems such as data consistency when using persistent memory.This brings challenges to the design of shared logs for distributed systems based on NVM.Aiming at the above scenario,this paper investigates the hardware characteristics of currently available persistent memory modules and designs and implements NSlog,which is a highly parallel shared log and uses CRAQ as the replication algorithm.NSlog takes advantage of asynchronous user-space networks,introduces a single-refresh log design for persistent memory hardware,and provides two different read operations,supporting strong consistency and final consistency,providing more flexibility for clients to access log entries.Compared with the traditional shared log,the shared log in this paper has the advantages of high throughput,low latency and scalability.Experimental results show that NSlog provides high throughput for read frequent workloads.In addition,this article demonstrated that NSlog is much more scalable than the original chained replication protocol,with excellent performance even as the number of servers,threads,and different log entry sizes increases.