Key-Value Database Based On Memory Partition And Adaptive Mechanism

With its compact structure and excellent read-write performance,key-value databases are playing a key role in a growing number of scenarios,such as e-commerce,online games,search engines,and advertising.There are various implementation schemes for the data layout in the key value database.For example,redis and memcache store the data in the memory and index it with the hash table structure.Leveldb and Cassandra store the data on the hard disk and index it in LSM tree structure.Because many applications in the real world have more data than memory capacity and persistent key-value storage is very necessary,it is of practical significance to study leveldb of LSM tree structure.LSM tree is a kind of data structure that converts random writes into sequential writes,and saves a large sort tree and a small sort tree at the same time.The small sort tree caches the random write requests of users in memory,and when a certain size is satisfied,writes the order to the hard disk and merges with the large sort tree.When HDDS are used as storage media,the LSM tree can greatly improve the efficiency of data writing,because the performance of random and sequential writes is quite different.However,the LSM tree's periodic merge operations for guaranteed read performance increase write latency.In the layer 0 structure on the LSM tree in hard disk,you can solve this problem by setting up a key prefix partition to reduce the range of key values for each layer 0 file.On this basis,ssd-based Memory Partition And Adaptive LSM(Memory Partition And Adaptive LSM tree)is proposed,which is referred to as MPAA-LSM for short.The Partition is put into the Memory implementation to reduce the system call overhead caused by frequent file creation in layer 0 And the write magnification problem caused by the combination of multiple small files in the Partition.When persisting,SSD parallel write operation is fully utilized to improve the persisting efficiency.In addition,MPAA-LSM monitors the read-write heat in different zones and time periods,and then adaptively adjusts key parameters in real time to further improve the system's read-write performance.Taking SSD as storage medium,the optimization scheme of MPAA-LSM was applied and simulate the distribution of multiple read-write temperatures as input load.The comparison experiment with the 0th layer partition and the original leveldb showed that the optimization scheme of partition mechanism was significantly improved in random read-write performance compared with the original leveldb.At the same time,compared with the layer 0 partition,MPAA-LSM has slightly improved the performance of random read and write in the heat load without read and write,while has further improved the performance of read and write in the load with hot and cold reading and write characteristics on the time series and partition,with the write delay reduced by about 18% and the read delay reduced by 16%.