Research And Application Of Efficient Hash Index Method Based On Hybrid DRAM-NVM Memory

In recent years,with the rise of big data and cloud computing,the data that computer systems need to process in real time has exponentially increased,which has brought huge challenges to traditional computer storage systems,especially memory capacity has become a limit to the development of storage systems Bottleneck.Compared with disks,DRAM can provide data access performance with low latency and high bandwidth,but at the same time it also faces performance problems such as high price,low integration,power loss,and high energy consumption.Faced with these challenges,the new type of non-volatile memory(Non-volatile memory,NVM)has brought new opportunities.Non-volatile memory not only has the byte addressing ability of DRAM memory and fast access characteristics,but also has the advantages of power-off non-volatile,high storage density,and low static power consumption.These characteristics enable NVM to form a hybrid memory structure with DRAM,and even replace DRAM as the mainstream memory storage medium in the future.In the memory system,the index structure,as the basic structure of data access,is essential to improve the efficiency of data access and memory space utilization.Among the commonly used index structures,the hash index has a constant search time and is currently the most efficient index structure for single-point query.However,the current hash index structure is only optimized for NVM performance,and does not give full play to the advantages of the hybrid DRAM-NVM memory system: First,there is still a certain performance gap between NVM and DRAM,which is reflected in that the read and write latency of NVM is higher than that of DRAM,and the read and write bandwidth is also far behind DRAM.This makes the hash index structure based on NVM design.Has higher index latency and overhead.The hybrid memory system can make use of the advantages of DRAM and NVM to complement each other,store metadata in DRAM as much as possible,filter unnecessary NVM reads and writes,and accelerate data access efficiency.Second,in a real industrial environment,hot data needs to be accessed frequently.If each search process needs to access the NVM to obtain these hot data,it will cause a long search delay and occupy too much NVM bandwidth.Using DRAM cache to store hot data sets in DRAM can effectively improve the performance of the hash index,but there is currently no targeted hash index caching strategy research,and traditional caching strategies cannot be well applied to the hash index structure..Therefore,this paper proposes an efficient hash index based on hybrid DRAM-NVM memory to solve the above problems.The main research contents and innovations are as follows: First,in view of the performance gap between NVM and DRAM,we propose a hash index structure based on hybrid DRAM-NVM memory,which fully decouples data and metadata,and extracts lightweight and frequently accessed metadata from NVM Come out,integrate these metadata into the designed data structure,and store the data structure in DRAM with better read and write performance,filter unnecessary NVM reads and writes as much as possible,and reduce data consistency guarantees and concurrency control.The overhead that comes.Second,for the hot search problem in actual workloads,we designed the hot table cache structure and proposed the RAFL caching strategy.Through cache management,make full use of DRAM cache frequently accessed hot data,monitor and dynamically update hot data sets in real time,take advantage of DRAM’s low latency and high bandwidth,improve data indexing efficiency,reduce NVM’s read and write bandwidth burden,and efficiently cache The replacement strategy also avoids the buffer jitter caused by frequent data swapping in and out of the hot table,and reduces the overhead of buffer replacement.