Research And Optimization Of Key-Value Store Read And Write Performance

The storage engine is the core of the storage system, and the reading and writing performance of the storage system depends on the performance of the storage engine.In this paper, the research is mainly on two main stream storage engine of Key-Value Store which are the LSM-Tree storage engine and the hash storage engine and analyzes the representative system’s reading and writing characteristics of the hash storage engine and the LSM-Tree storage engine. LSM-Tree is characterized by sequential writes while the hash storage engine focuses on the efficiency of the memory index.Based on the structure of LevelDB, this paper presents the structure of sLSM-Tree(Segmented-Index based LSM-Tree) by analysis of the index and hard disk storage of the two major storage engines. 1) In the memory index section of the storage structure, a segmented index structure is introduced which is used to resolve conflicts brought by adding hash storage memory index structure for LSM-Tree. Namely, improve the indexing speed by introducing the prefix tree index and hash mapping index. While to avoid data reading by layer index, reduce the pressure caused by the frequently change of physical address which is created by the merge operation when writing the data. 2) To further improve the efficiency of Key query, this paper proposes an optimization of the dynamic array Bloom filter which is based on the traditional Bloom filter algorithm adapted the memory index structure. Namely, reducing the read and write time of the entire array by dynamically setting the grouping factor of the Bloom filter array. Persistence structure is still using the Append-Only log structure to maintain the original LSM-Tree high speed write. Flash based state drive can provide a high read and write performance. However, the traditional LSM-Tree compression merge operation is for mechanical hard disk. In this paper, the corresponding optimization is introduced by the structure and characteristic of compression merge operation on SSD.In this paper, comparative experiments of the new method of segmented index and dynamic array type Bloom filter optimization were carried out. From the experimental results, compared with the LevelDB which adopts traditional LSM-Tree storage engine, sLSM-Tree has a significant performance improvement in memory index and hard disk read and write. Finally, discuss the sLSM-Tree further promotion based on experimental data and further analysis.