Research And Implementation Of A Key-Value Storage System Performance Optimization Technology Based On Hybrid Storage Structure

The NoSQL database represented by key-value storage system provides a high performance,high scalability as well as more flexible data storage,which can meet massive data storage requirements of big data environment.LSM-Tree(Log Structured Merged-Tree)is one of the mainstream storage engines for key-value storage system,performing well random write performance.However,LSM-Tree also brings serious write amplification when system triggers compaction,which severely restricts system write performance improvement.NVM(Non-Volatile Memory)is a new storage medium with read and write performance close to DRAM.Therefore,research on the key-value storage systems based on NVM and SSD hybrid storage structure has important practical meanings.The thesis proposes a Range Key-Value(RangeKV)system with SSD and NVM hybrid storage,aiming at LSM-Tree structure write amplification problem of key value storage system.Use multiple RangeTab structures on NVM to organize key-value pairs in Level 0,and appending write data to RangeTab,which can improve write efficiency.Besides,RangeTab structure increases level 0 capacity,which indirectly shorten LSM-Tree level number,reducing overall compaction times for key-value storage system.Building hash index for RangeTab can improve the lookup performance of RangeTab,which is able to quickly locate key-value's physical address and lead to less time to access to NVM.What's more,double buffer structure design for compaction make contributions to relieving system stall.In order to decrease write amplification for compaction,the RangeTab data with the lowest write amplification ratio is preferentially selected to participate in the compaction to enhance the efficiency of compaction and further improve system write performance.Based on open source plateform RocksDB,the thesis implements RangeKV storage system,and experiments comparisons are done among RangeKV,RocksDB as well as NoveLSM.Test results show that RangeKV can improve random write performance 4.5x to 5.7x compared to RocksDB-SSD.RangeKV can reduce more than 50% compaction times,and drop average compaction data size about 40% over RocksDB.Also,the overall system write amplification ratio is only about 25% of RocksDB.In addition,RangeKV still owns a significant performance advantage compared to NoveLSM.