Research On Optimization Of A Key/value Storage Engine Based On LSM-Tree

In the face of the growing speed of the data and the increasingly diverse forms of data organization,NoSQL database has been widely recognized and used,with the characteristics of fast speed,good compatibility,easy to spread and so on.At the same time,the proportion of write operations is increasing,how to optimize the write speed of NoSQL database and make it provide better write performance has already become the focus topic in the field of data storage systems.Among all the popular KEY/VALUE storage models,the write-optimized data structure LSM-Tree(Log-Structured Merge-tree)and its variants becomes the core data structure and has been widely used in the mainstream KEY/VALUE storage databases.However,due to the defects of the compaction scheduling mechanism of LSM-Tree data structure,there is a serious problem,named write amplification,in the running process of the system,which greatly affects the write performance of LSM-Tree and throughout of overall system.A optimization is found to solve the defects of the compaction scheduling mechanism of LSM-Tree data structure through a series of scientific and rigorous experiments,which is called the Time Slice Rotational Scheduling Mechanism(TSRS)and developed based on RocksDB.Through a series of experiments,it is proved that this optimization can make sure that the file number on each component of LSM-Tree grows smoothly through fair compaction and decrease write amplification caused by the excessive number of file on the component,and then improve the write performance of LSM-Tree.TSRS has fundamentally sloved the problem of excessive write amplification of LSM-Tree.Compared with RocksDB,the write performance improvement can reach more than 4 times.TSRS can be applied to any NoSQL storage engine based on LSM-Tree.