Research On Optimization Scheme Of Read And Write Performance Of Key-value Storage System Based On LSM-Tree

The storage system with The Log-Structure Merge Tree(LSM-Tree)as the engine provides storage services for today's data-intensive applications.LSM-Tree achieves good write performance by converting random writes to sequential writes and compresses the written data to improve read performance and reduce storage space usage,but the compression operation will bring about write amplification and write pause problems.At present,the research on the write amplification problem is relatively mature,but solving the write amplification problem has limited improvement in the overall performance of the system.Therefore,it is necessary to optimize the write stall problem to further improve system performance.In response to this problem,this paper analyzes the causes of write stalls in the memory and disk parts of the LSM-Tree system,and proposes a solution to improve the performance smoothness of the system,thereby further improving the overall performance of the system.The main research contents and plans of this paper are as follows:(1)This paper proposes an optimization scheme for multiple Immutable Memtables modules.This scheme alleviates the blocking problem during Memtable conversion caused by the full space of the module by expanding the number of Immutable Memtable modules.Improved system writing performance.In addition,the scheme opens up more storage space in the memory,caches more data,obtains a higher memory read hit rate,and improves the read performance.Experiments show that this solution can improve read performance by 4%-65% and write performance by 15%-70%.(2)This paper proposes a scheme to optimize the contention of threads when multiple Immutable Memtables modules write data to disk.This solution alleviates the problem of multiple modules competing for a single thread by increasing the number of threads and allows each module to use an independent thread to read and write disk data to improve system performance.Experiments show that this solution improves the read performance by 12%-40%,and improves the write performance by 3%-11%.(3)This paper proposes a semi-ordered scheme to alleviate the writing stall caused by merge operations.This scheme optimizes the merging process by adding a semi-ordered layer between the zeroth layer(level 0)and the first layer(level 1)of the disk,allowing the two layers to be merged with the semi-ordered layer respectively,reducing the direct merger of the two layers.,in order to improve the merge efficiency,greatly alleviate the LSM-Tree write pause problem,and improve the system write performance.Experiments show that the writing performance of the system is improved by 8%-18%.