Optimization Design And Implementation Of Key-value Separation Store System Without Garbage Collection

Key-value store is an important part of modern storage systems.Since the LSM-tree data structure is optimized for random writes on disks,the write performance of key-value store systems is greatly improved,so LSM-tree-based key-value store becomes mainstream.In order to overcome the phenomenon that the LSM-tree still produces high read amplification and high write amplification,it has further developed into a key-value store system using key-value separation.However,key-valued storage systems that use key-value separation trigger garbage collection(GC)operations frequently under update-intensive workloads,making them unable to achieve higher performance.Aiming at this problem,this paper designs and implements a key-value store system without GC.The system realizes in-place update through effective management of invalid data,thus eliminating garbage collection operations in the storage process and avoiding frequent rewriting of valid data in the storage system.The system can reduce system write amplification and improve system performance.First,the invalid data index is collected,managed and reused to update and recover the invalid data in place to eliminate the GC process and avoid the overhead caused by GC.Secondly,the invalid index management module is implemented,and the producer-consumer model is built to optimize the module so that it can flexibly and efficiently realize the management and reuse of invalid data in the system,which provides a basic guarantee for the removal of GC operation in the system.Finally,the logic design provides the guarantee of update consistency for the in-place update writing mode of data.At the same time,by adding and managing the operation log,the data recovery after the system crash is realized and the data loss is avoided.The tests show that the key-value separation store system without GC can effectively reduce the amount of system write.Under the update-intensive workload,compared with the existing key-value separation store system Wisc Key and Hash KV,the system can reduce the amount of write by 30%~50%,thus reducing the system write amplification.The system removes the GC process and improves the write performance.At the same time,the system performance will not be affected by the size of the reserved space and the key-value data.