The Design And Implementation Of Low-latency Distributed Key/Value Storage Based On Non-volatile Memory And RDMA

In modern large web services,In-memory key/value caches or stores are deployed to provide high throughput data access at a low latency.As data in DRAM will be lost after power failure,it's usually flushed to secondary persistent storage(e.g.SSD,RAID)to offer durability(e.g.Redis),and replicated to several independent servers to tolerate individual failure.In this paradigm,network and secondary storage's bandwidth and latency become the bottleneck that many system have to sacrifice consistence guarantee to eliminate IO latency.Two emerging technologies,non-volatile memory(NVM)and remote direct memory access(RDMA),are promising to overcome this limitation.The RDMA technology provides remote direct memory access in user space(kernel bypassing)without invoking peer CPU.And NVMs have high access speed,low cost and byte addressability.To reduce the latency of network and persistent storage,we present and design of a NVM-based low-latency distributed key/value store by leveraging RDMA.By Utilizing durability of NVM and direct memory access of RDMA,we made fast synchronous primary backup replication(PBR)without sacrificing consistence.In write requests,we record updates(data and metadata)at primary server's local NVM and direct replicate them to the same relative addresses of slaves by RDMA writes.In this paper,we have 4 main contributions:· Design and implement a prototype of low latency distributed key/value store based on NVM and RDMA:By utilizing the high speed,high bandwidth and durability of NVM,combined with low latency of RDMA,it provides data access in micro seconds.Especially,we designed low latency primary backup replication,making synchronous replication possible.· Optimize RDMA verb for different scenarios:We analyse scenarios in distributed storage system that RDMA technology applied to.We choose carefully choose different verb for each scenario and reduce request latency without sacrificing scalability.· The design of fast primary backup replication:We design fast,synchronous primary backup replication with durability of NVM and low latency of RDMA.Our design provides better consistence guarantee,compared to current distributed storage systems.Also,we analyse the bottleneck of the whole replication processing,and handle possible failures.· Evaluation of request latency:We evaluate request latency on our system.Especially,we analyse each module's latency in different requests separately.Our system provides6?s to 8?s request latency,and the replication latency is between 2?s and 3?s.Last,experiments show that our system has one-fifth latency of Memcached.