Research On Key Techniques Of Hybrid Memory Management For Big-Data Application

On the era of Big Data,the massive growth of data volume and high performance of data processing highlight the performance bottleneck of disk-centric computer storage systems.Compared with HDD or SSD,DRAM can provide high performance of data access,but it faces challenges of high cost per bit,low density,volatibility,and high energy consumption.With the development of Non-Volatile Memory(NVM)technology,its high density,non-volatibility and access speed like-DRAM and so on,which brings the opportunities to improve this dilemma,however,NVM has limited write endurance and long write latency,which makes NVM totally replace DRAM impracticable,while a more exciting idea is to to use both NVM and DRAM to construct hybrid memory systems.In this context,first,one appropriate NVM space management mechanism can provide the convenience for user program to using non-volatile memory space;Secondly,sharing NVM space in the cluster can provide more storage space to meet the large capacity of big-data applications;Finally,the existing in-memory data management is based on the characteristics of DRAM,without elaborate design for utilizing NVM advantages and improving its deficiencies.Based on these aspects,this dissertation first studies the mechanism of Non-Volatile Main Memory management in single machine and distributed environment,respectively;and then focuses on the data management optimization combining with NVM and RDMA technology.The main contributions of this dissertation are as follows:(1)Proposing a wear-conscious main memory allocator for NVMNVM has the merits of byte-addressability,fast speed,persistence and low power consumption,which make it attractive to be used as main memory.Memory allocator,as a direct way to get memory blocks for user applications,and wear-conscious memory allocator simplifies the use of Non-Volatile Main Memory(NVMM)in user applications.However,traditional memory allocators designed with in-place data writes are not appropriate for NVMM due to its limited write endurance.In this work,first,we quantitatively analyze the wear-oblivious of DRAM-oriented designed allocator—glibcmalloc and the inefficiency of wear-conscious allocator—NVMalloc.Then,we propose WAlloc,an efficient wear-aware manual memory allocator designed for NVRAM:(1)decouples metadata and data management;(2)distinguishes metadata with volatility and redirects the data writes around to achieve wear-leveling;(3)selectively allocates DRAM write buffers for the allocated NVM blocks;(4)redesigns an efficient and effective persistent NVM copy mechanism,bypassing the CPU cache partially and prefetching data explicitly.Finally,experimental results show that the wear-leveling of WAlloc outperforms that of NVMalloc about 30%and 60%under random workloads and well-distributed workloads,respectively.Besides,WAlloc reduces the average data memory writes in 64 bytes block by 1.5 times comparing with glibc malloc.(2)Designing the remote memory extension system based-on RDMA-able NVMConsidering the limited density of DRAM,traditional local memory extension is bounded by slow storage media(HDD/SSD)due to storage IO performance bottleneck and limited storage capacity of single machine,remote memory extension has become possible with RDMA technology providing direct access to remote NVMM.In view of this,we design a remote non-volatile memory extension system,Pyramid,which aggregates non-volatile memory spaces on other nodes in NVM-based cluster by RDMA technology.First,we revisit local memory extension mechanism and the performance opportunity of remote NVMM access by comparing it with the new technologies of storage-NVMe-SSD and PCM-based SSD.Then,we detail the design of Pyramid.It includes establishing a mapping relationship with remote virtual memory address to achieve remote NVMM access,adopting NVMM allocation mechanism with remote page fault,improving the performance of remote memory allocation with dynamic synchronization based-on RDMA atomic operations,attaching the local DRAM as cache layer for remote NVMMs to speed up the data read/write performance and absorb NVM data write,and providing the basic memory allocation and access interfaces to enhance the integration and usage compatibility in application system.Finally,the experimental results show that Pyramid provides remote memory allocation performance as low as 11 microseconds to 15 microseconds.(3)Designing and implementing in-memory key-value system based-on NVMStorage system based-on key-value model has the benefits of good scalability,which makes them widely used in big-data processing systems,considering the performance advantage of DRAM,so the design of in-memory key-value system has become a hot research area.However,the electrical characteristic and limited density of DRAM make the storage IO inevitable;in addition,network performance also limits the overall performance of in-memory key-value systems.With the combination of RDMA technology and NVM,it brings new opportunities for the design of in-memory key-value system,but limited write endurance of NVMM and the new communication mechanism of RDMA are the roadblocks to the way of achieving high performance in-memory key-value systems.Hence,we propose an in-memory key-value system based-on NVM,which uses RDMA for remote access and also supports the management of traditional memory.Craftsman incorporates several recipes.These techniques include:1)buffering remote memory writes into a communication cache memory to eliminate direct remote memory writes to data memory area;2)dividing the communication cache memory into RDMA write-able and read-able memory zones to reduce the possibility of data corruption due to the stray memory writes and caching data into RDMA-readable memory zone to improve the remote memory read performance;3)adopting remote out-of-place direct memory write mechanism to achieve high performance of remote read and write.Experimental results imply that Craftsman provides better performance than Memcached.As an example,in the case of read-intensive workloads,the data access latency of Craftsman is about 18%~72.4%better than that of RDMA-based Memcached.(4)Proposing one variant B+-Tree based on remote accessible NVMMThe data consistency of B+-Tree based on NVMM is achieved by explicitly manipulating the CPU cache so that the stored data in NVM can survive the system failure.However,the extra memory writes caused by elements sort in B+-Tree speed up the write endurance of NVMM.Besides,long write latency of NVM hurts the concurrent access performance of B+-Tree.In the light of these,we propose one variant B~+-Tree,Rio,based on NVMM with RDMA technology.First,the internal nodes of B~+-Tree are put into DRAM and persists its leaf nodes into NVMM;Secondly,taking advantage of the ability of direct access to remote NVMM with RDMA,offloading the persist work of leaf nodes to clients to simplify the server's processing logic and increasing its processing throughput;Then,organizing elements in the leaf nodes in log-structured way to reducing extra memory writes and delaying the splitting process of leaf nodes to reduce memory write;Furthermore,attaching one index leaf node to each data leaf node to maintain range scan performance;Finally,combining RDMA atomic operations with local atomic operations to achieve remote concurrency control easing the server lock competition.The experimental results show that the throughput of Rio is increased by about 56%and the end-to-end operation delay is reduced by about15%compared with existing B+-Tree based-on hybrid memory.