Font Size: a A A

Research On Optimization And Implementation For LSM-Tree-based Key-Value Stores

Posted on:2020-06-05Degree:MasterType:Thesis
Country:ChinaCandidate:Q FengFull Text:PDF
GTID:2428330599958590Subject:Computer technology
Abstract/Summary:PDF Full Text Request
NoSql databases represented by Key-Value(K-V)databases can provide high performance,high scalability and high flexibility data processing sevices for large data applications.Log-Structure merge tree(LSM tree)is one of the mainstream storage structures in Key-Value storage systems,It converts random writing into sequential writing,which can provide high write performance.However,Compaction operations within LSM tree can cause significant Write Amplification problems,especially in the case of large KV size or high update frequencies.WiscKey design for reduces the size of LSM tree by storing key and value separately,and alleviates the problem of Write Amplification to a certain extent.However,in update-intensive workloads,invalid values in value storage files will increase the disk capacity and increase the cost of garbage collection,which will degrade the system performance.In order to solve the problem of low performance in update-intensive workloads,this paper proposes a key storage system based on LSM tree structure and key separation.Through a set of mechanisms,the thermal data sensing scheme can effectively identify the data with update-intensive and store it in the hot data cache,which can greatly reduce disk overhead,and also reduce the number of LSM tree merging operations and the space capacity of invalid data in key value separation storage.This paper implements a prototype of key value storage system with hotness awareness,and tests the system in update-intensive workloads and compared with LevelDB.The test results show that in Zipf distribution,the storage system with data Hotness awareness can effectively put a large number of update operations into memory with relatively small buffer capacity,and its hit rate is more than 80%.In big value size workloads,the read-write performance is significantly better than LevelDB.
Keywords/Search Tags:Key-Value store, Log-Structure merge tree, Key-Value Separation, Hotness awareness
PDF Full Text Request
Related items