Research On NVM Dynamic Memory Allocator For Different Application Scenarios

In the era of big data where global data volumes are exploding,traditional storage system architecture has become a bottleneck.The emergence of NVM(Non-Volatile Memory)is expected to solve the performance gap between internal and external memory in traditional storage systems and meet the memory access requirements of data-intensive applications.Under the architecture of DRAM(Dynamic Random Access Memory)and NVM,because NVM needs to meet different memory requirements in different application scenarios,the traditional user-level memory allocator is no longer applicable,and it is necessary to redesign the memory allocation for NVM characteristics.The existing NVM memory allocator does not fully consider the characteristics of NVM media,and the design of the data structure cannot meet the memory requirements of NVM in different application scenarios,which makes the metadata overhead larger and causes problems such as low memory utilization.Aiming at different application scenarios of NVM,a high-efficiency memory allocator under comprehensive application scenarios,POMalloc,is designed and implemented to fully utilize the advantages of high performance and durability of NVM in non-volatile scenarios,and large capacity and low cost in volatile scenarios.In response to the application scenario requirements of non-volatile memory services,a coarse-grained memory management scheme was designed based on the AVL tree to avoid the NVM write performance degradation caused by frequent metadata operations.At the same time,a data recovery mechanism that manages the allocation status by the application itself was designed to avoid memory permanent leakage.In response to the requirements of volatile memory service application scenarios,the bitmap structure is used to finely divide the memory blocks to solve the problem of low memory utilization caused by frequent small-byte memory allocation.At the same time,the driver operating mechanism is designed to provide applications lightweight mixed memory allocation.The actual application scenarios were tested on the Redis and rocksdb storage systems.The results showed that under non-volatile scenarios and volatile scenarios,the throughput of the Redis system increased by 10.6% and 19.9% respectively,and the DRAM memoryfootprint was reduced compared to the original system 96.3% and 93.7%,and rocksdb data memory usage is only 60% compared to innodb,which improves system performance while greatly reducing memory usage.