Research On Optimization Of Persistent Key-value Storage System Based On SSD-NVM

The key-value storage system based on the Log-Structured Merged Tree(LSM-Tree)has received more and more attention due to its great read and write performance,and has become a hot spot in recent research on storage system.However,the current LSM-Tree key-value storage system cannot efficiently handle skewed workloads,and data will be written to the persistent device frequently,preventing insertion of key-value pairs.At the same time,there is a problem of the system's write amplification,and there is room for improvement in the efficiency of metadata reading.In view of the above problems,optimizations on the LSM-Tree key-value storage system that takes solid state drives(SSD)as the main storage device and introduces nonvolatile memory(NVM)are proposed.First,in order to efficiently handle skewed workloads that updated frequently,a persistent hot data cache management strategy is designed to use NVM to cache "hot" data that updated frequently.The strategy can avoid flushing data to SSD frequently,thereby reducing the system's write latency.In addition,a hybrid index structure has been adopted inside the cache to ensure efficient insertion and read of data,without sacrificing the system's query function.Secondly,in order to reduce the system's write amplification,a classification and merge strategy based on overlap ratio of file is proposed.Files involved in the compaction are classified and merged in parallel,to reduce the amount of data rewriting and control the space amplification.Finally,considering that the overhead of metadata reading will affect the system's read performance,a separate storage strategy for file data and metadata is designed,and the file data is stored in SSD while the metadata is stored on fast NVM.In addition,in order to reduce the overhead of metadata reading,a special library is used to access metadata directly.The above optimization technologies are used to implement the prototype of SNKV(SSD-NVM Key-Value)based on LevelDB.The test results show that,compared with NoveLSM which is the international advanced key-value storage system,SNKV can improve write performance by 56%?5.5x under skewed workloads,while it can reduce write amplification by 27%?33% and improve write performance by 65%?72% under nonskewed workloads.In addition,as the number of read requests increases,SNKV's read performance can be improved by about 2x compared to NoveLSM.