Research And Implementation Of A New Hybrid Tree Key-value Store System

With the advent of the 5G era,the data generated by devices such as smart terminals and sensors is growing exponentially,and the demand for cloud infrastructure is constantly expanding.As a representative of unstructured databases,key-value storage systems play a pivotal role in data centers.One of its main storage engines is the log-structured merged-tree(LSM-tree).However,in actual tests,it is found that due to the overlap of the L₀ layer key range of the LSM-tree structure and the frequently triggered L₀ layer capacity control mechanism,the merge operation of the LSM-tree can cause periodic fluctuations in the write performance of the key-value storage system.Aiming at the problem of periodic fluctuation of the write performance of the key-value storage system,a new mixed tree key-value storage system BigMixDB solution is proposed.First,a new mixed tree index structure BigMixTree is designed,which adopts a two-layer structure(index frame layer and sub-B+tree layer.Through the shunting function of index frame layer,the Memtable and the hard disk L₀ layer data of the LSM-tree are uniformly organized in units of sub-B+trees,avoiding the overlap of the key range between the files in the L₀ layer.Secondly,a new key-value storage system named BigMixDB is designed based on the BigMixTree index structure,which uses BigMixTree to convert the original large data merge operation of L₀ and L₁ layer into multiple small data merge operations,reducing the time that the system is in a write suspended state.In addition,because BigMixTree contains memory Memtable and hard disk L₀ layer data,the query level of the BigMixDB is reduced,and the read performance of the BigMixDB is improved.Finally,based on the BigMixTree,the process of BigMixDB merge operation is redesigned to ensure that the merge process between the BigMixTree and L1 layer is smooth and the amount of data in a single merge is small,so as to solve the periodic fluctuation of write performance of the key-value storage system.Based on the open source LevelDB,the BigMixDB prototype is implemented,and the comparison between the BigMixDB and LevelDB is done on the Intel server and Huawei cloud server.The evaluation results show that BigMixDB can effectively minigate the write performance fluctuations,and the fluctuation degree is only 33.56%of LevelDB.At the same time,the random write performance of BigMixDB is 1.64?2.06 times of LevelDB,while the random read performance is 1.10?1.75 times of LevelDB.