| Recently,the emergence of Non-volatile Memory(NVM)and Remote Direct Data Access(RDMA)technologies has attracted increasing interest in building distributed systems with them.The volatility of the cache in the RDMA Network Interface Card(RNIC)makes how to ensure the persistence of the remote data a major challenge.Moreover,the Data Direct I/O(DDIO)technology exacerbates this challenge.The read-after-write solution has better performance,but it must disable the DDIO of the remote system,which will reduce the performance of other applications without persistence requirements.Other solutions do not have to disable the DDIO but require the remote CPU to flush the data and thus bring additional performance overhead.Therefore,the existing solutions can not meet the requirement of various applications at the same time.Regarding the issue above,a mechanism supporting fine-grained DDIO control is proposed for remote data persistence.Firstly,the problem of remote data persistence caused by DDIO is analyzed in detail,the performance differences of existing solutions are demonstrated through experimental tests,and then some design guidelines are presented.Secondly,for the scenario that only needs to persistently write data to the remote NVM without remote CPU processing,a Linux kernel module is implemented to assist user-mode programs to allocate uncacheable NVM to bypass DDIO.For the scenario where data needs to be persistently written to the remote NVM and the data needs to be processed by the remote CPU later,it is proposed to dynamically control the state of DDIO at runtime so that applications can dynamically select different persistence options according to different data granularities to maximize the persistence efficiency.Finally,extensive experiments are conducted to compare this persistence mechanism with traditional solutions under a unified environment equipped with RDMA and NVM.Experimental results show that fine-grained control of DDIO improves the throughput by 84.2% compared with the method that needs remote CPU flush,and reduces the memory bandwidth occupancy of other applications without persistence requirements by 30.7%compared with the read-after-write method. |
