Research On Read And Write Optimization Technology Of LSM-tree Based On NUMA Architecture

With the rapid growth of the amount of data in various fields and the increasingly complex data types,the proportion of unstructured data in the form of images,audio,and video has gradually increased.These phenomena have raised comands about the read and write performance of non-relational databases such as key-value storage and higher requirement.LSM-tree(Log-Structured Merge-Tree),as a layered,ordered,disk-oriented data structure in key-value storage systems,has been widely used in various big data scenarios.At present,processors based on the ARMv8 architecture are more and more widely used in cloud computing,AI,5G,and other fields,and are gradually being used to build new computing platforms with high performance and low power consumption.For example,Huawei’s Kunpeng 920 processor is used for the Tai Shan server.However,the Kunpeng 920 processor has the phenomenon that the number of cores increases,and the non-uniform memory access(NUMA)effect is enhanced.The LSM-tree itself also has problems such as write blocking and read amplification.These phenomena limit the resource utilization and scalability of LSM-tree under the Kunpeng 920 processor further affect the read and write performance of the storage system.Faced with the problems mentioned above,this paper analyzes the Kunpeng 920 processor with the open-source data storage engine Level DB as the research object.This paper focuses on the read and write process of Level DB and the characteristics of NUMA architecture.This paper analyzes the bottleneck of system read and write performance,and adopts the idea of multi-thread management and cache optimization in each NUMA node to adapt it to the application scenario of Kunpeng 920 processor.The main research contents and contributions of this paper include:(1)LSM-tree multi-threaded write strategy based on NUMA-AwareThe NUMA architecture in Kunpeng 920 causes delays when each processor core accesses the memory of a node other than its own.When the resources in the memory are insufficient,Level DB’s own Arena(memory management tool)may has an unreasonable allocation area,triggers cross-node memory access events.In addition,Level DB has a write blocking problem in multi-threaded mode,and cannot effectively utilize the multi-core feature of the processor.In order to solve these two problems,this paper proposes a NUMA-Aware-based LSM-tree multi-threaded write strategy.This scheme deploys memtable and immutable memtable on every NUMA node for partition writing,designs an adaptive logical CPU binding strategy and modifies Arena,and proposes an actual writing thread and a virtual writing thread(to monitor the load situation,pre-filled Write Batch)to increase memory usage and write efficiency.The experimental results show that the writing performance of the scheme in the dual Kunpeng 920 processor on-chip system improves with the increase of the number of NUMA nodes,has good scalability,and can adapt to different sizes of key value data.The average performance in sequential writing is improved by 125.3%,and the average performance in random writing is improved by 14.1%.In addition,the scheme maintains almost the same reading performance and can increase the read hit rate in memory.(2)Double granularity cache scheme based on NUMA nodeWhen LSM-tree looks for key-value data,it needs to check multiple SSTable files and generate multiple I/O accesses,thus causing the problem of reading amplification.The traditional cache scheme will have update invalidation,and the cross-die and cross-chip memory access events under the NUMA architecture will also affect the read performance.To solve these problems,this paper proposes a double granularity cache scheme based on NUMA nodes.This scheme sets up Filter Table,a data structure for recording SSTable metadata information for each NUMA node,and divides the level of Level into upper and lower parts.SSTable in Level 0 – Level 2 uses Filter Table to manage metadata information by partition.The other Level uses the LRU cache to record frequently accessed SSTable information.The experimental results on the Tai Shan2280 platform show that the cache scheme reduces the cross-die and cross-chip read and cross-die and cross-chip write operations events by 10.4% and 7.3%respectively,improves the random read performance by 13.4%-70.1%,and can adapt to key value data of different data sizes.