Resource Abstraction And Data Placement For Distributed Hybrid Memory Pool

Today's big data applications require more and more memory for high-performance data processing.The traditional distributed shared memory pool based on Dynamic Random Access Memory(DRAM)provides large-capacity shared memory resources for applications,but suffers extremely high monetary cost and energy consumption.Emerging byte-addressable Non-Volatile Memory(NVM)offers higher density,lower cost and nearzero static power consumption than DRAM,at the expense of lower access performance and limited write endurance.In recent years,a distributed hybrid memory pool composed of DRAM and NVM with large capacity,low cost and high performance has attracted wide attention.Due to the performance difference between DRAM and NVM,the memory management strategies in traditional memory pools cannot fully utilize the characteristics of different media when applied to distributed hybrid memory pools.At the same time,the hybrid memory management strategies on a single server do not support data placement across servers and load balancing,and are not applicable to distributed hybrid memory pools.Therefore,it is still an open problem on how to manage hybrid memories efficiently in a distributed environment.In order to efficiently use DRAM/NVM hybrid memory resources in a distributed environment,we propose Alloy,a memory resource abstraction and data placement strategy for a distributed hybrid memory pool,which enables applications to transparently use hybrid memory resources in the pool through simple application programming interfaces.Remote direct memory access technology is used to achieve communication message transmission and memory access between nodes in the pool to reduce communication overhead.Alloy monitors the use of cluster resources and the memory access behavior of objects in real time through memory resource aggregation and abstraction mechanism,and classifies objects through memory hotness identification model.In order to make full use of memory media with different performances,Alloy uses a hotness-aware data placement scheme,which combines data replication,write merging and global greedy based data migration technologies to place objects with different access characteristics to improve application performance and reduce the cost of the memory pool.Experimental results show that Alloy can significantly reduce the DRAM usage in the distributed hybrid memory pool by up to 95%,while reducing the total memory access time of application by up to 57% compared with the state-of-the-art approaches.