Research On LSM-tree Optimization Technology Based On Non-Volatile Memory

The data model of key-value pairs is widely used in data-intensive applications such as e-commerce,social networks,and web searches due to its high scalability.As a storage tool for key-value pairs,the key-value storage engine provides Fast,durable data storage service.LSM-tree is currently the most widely used storage engine architecture,which has high write performance,but the problems of read-write amplification and write pause have not been well solved in the current computer system.In recent years,a A new type of storage medium — non-volatile memory,is expected to use it to solve the above problems.Past studies have shown that the problems of read-write amplification and write stalls are closely related to the L0 layer data organization of LSM-tree.This paper will study the storage architecture of LSM-tree based on non-volatile memory to solve the problems of read/write amplification and write stalls.At the same time improve its read and write performance.This paper redesigns PMem-LevelDB,a key-value storage engine based on nonvolatile memory,based on LevelDB.The main innovations and key technologies of PMemLevelDB include:(1)An efficient non-volatile memory allocator suitable for PMemlevel DB is designed to support instant recovery and ensure data consistency,while supporting media wear leveling.(2)Based on the DRAM-NVM-SSD/HDD architecture,transfer the L0 data to the non-volatile memory,and design a unified L0 jump table with the pre-write log,and reorganize the L0 layer data from a global perspective.(3)According to the data organization characteristics of non-volatile memory,design the data merging algorithm,cache mechanism and fault recovery algorithm.In order to verify the effectiveness of the PMem-LevelDB scheme,a micro-benchmark comparison experiment and a macro-benchmark comparison experiment are compared with the original LevelDB.The experimental results show that PMem-LevelDB doubles the write amplification factor compared to LevelDB,and PMem-LevelDB is 6 times faster than LevelDB on average in terms of random write speed.In terms of query performance,PMem-LevelDB point query is slightly faster than LevelDB,while range query is 3 times faster on average,and there is no case where the write throughput suddenly drops to 0.The paper also analyzes these phenomena and fully verifies The feasibility of the scheme is confirmed,and finally the scheme is summarized and the future prospect is given.