Research And Implementation Of Key-value Storage System Based On Optane SSD

The key-Value databases are widely used due to its simple structure,fast query speed,high reliability and high scalability.Nowadays,the performance of Key-Value storage engine is increasingly demanded by all kinds of applications while traditional hard disk performance is too slow to meet the requirements.Intel launched the Optane series products inn order to solve the problem of slow hard disk performance.Optane SSD,as a new-type storage media disk product,has good read and write performance,and meanwhile enable random read and write performance to keep the pace with the sequential read and write performance.The traditional Key-Value storage engine uses the structure of lsm-tree to convert random write to sequential write cannot get high returns and is no longer suitable for the new Optane SSD.KVell uses a combination of memory and Optane SSD to record index information in memory and data in solid-state disks to obtain good performance,but when the amount of data is too large and index information cannot be completely stored in memory.The decline of Kvell's perform is obvious and the tail is serious.Based on the excellent random read and write performance of Optane SSD,this paper abandoned the traditional LSM-Tree data structure and designed a Bag Tree data structure with data organization similar to the B + tree.Bag Tree organizes data in the way of tree does,expanding its capacity by adding bag nodes to the leaf node ensures reasonale use of leaf node space and controling the number of intermediate nodes in the tree structure flexibly,and as a result,Bag Tree can be used normally when the memory is small.Designing a private multi-layer cache for Bag Tree and set elimination levels and elimination algorithms according to cache queues make sure the middle nodes of the Bag Tree tree structure are resident in memory.To improve the efficiency of the storage engine.Bag Tree the leaf nodes and backpack nodes are cachereplaced by multi-layer LRU.The test results show that Bag Tree's random read performance outperform Level DB by 30 times,Rocks DB by 20 times,KVell by 1.7times.The Bag Tree's random write performance is 3times compared to Level DB and 2times compared to Rocks DB,and it is slightly better than KVell.From the single-threaded single-queue test in the YCSB test,the read performance of Bag Tree is 2times compared to Level DB and Rocks DB,and the write performance is 6times compared to Level DB and Rocks DB.The multi-threaded multi-queue test in the YCSB test shows that the performance of Bag Tree is up to 1441 KIOPS,outperforms KVell does by 2times,and the maximum latency is also better than KVell's.