Research On RDMA-Based Distributed Shared Memory In Virtualization Systems

With the rapid development of the Internet and the steady growth of data collection and storage capabilities,the amount of data accumulated by humans to be processed and utilized has reached a level that traditional mainframes are unable to handle.Faced with the challenge of this "big data" problem,a large number of scale-out solutions,represented by distributed parallel computing frameworks such as MapReduce and Spark,were proposed,and gradually replaced traditional scale-up solutions.These systems achieve great scalability through partitioning and distributing data to a cluster of machines to process.But they also require users to learn and use their special programming models as well,giving users cognitive burden and migration cost of legacy softwares.Another important trend in the Internet era is cloud computing,which relies on virtualization technology to achieve the division,management and on-demand allocation of computer resources.On the cloud,a single virtual machine has limited processing power and thus is not capable of handling big data processing tasks.To overcome this limitation,one solution is to use virtual machines to form a cluster and run distributed computing frameworks on it,thereby breaking the bottleneck of the computing power of a single virtual machine.This paper proposes a distributed virtualization architecture that uses multiple physical machines to form a virtual machine and can be used as another solution.The architecture does not require users to change usage habits while implementing resource aggregation,avoiding the corresponding drawbacks of distributed computing frameworks.This paper focuses on the design and implementation of distributed shared memory in distributed virtual machine monitors,whose purpose is to implement memory virtualization.In order to implement and integrate high-performance distributed shared memory in a virtualized environment,this paper innovatively proposes the following mechanisms:(1)A memory synchronization protocol based on relaxed consistency and aimed for cooperation with paravirtualization.In distributed shared memory systems,the choice of synchronization protocol has a significant impact on performance,and most recent works are based on a relaxed consistency protocol.However,virtualization systems require that implementations of memory be completely transparent to guests,which constrains us to only use sequential consistency based protocols that are not able to achieve optimal performance.In order to solve this problem,this paper proposes a synchronization protocol that allows different vCPUs to be in different consistency modes.Used in conjunction with paravirtualization technology,the limitation of memory consistency imposed by virtualization systems can be effectively bypassed.(2)The integration mechanism of distributed shared memory and I/O virtualization.In a virtualization system,I/O virtualization and memory virtualization are not completely orthogonal,and DMA initiated by an I/O device accesses memory.In order to integrate DMA accesses with distributed shared memory,this paper proposes a proxy mechanism as well as a read-write lock mechanism for pages in distributed shared memory,which are used to deal with virtual devices implemented through and not through device passthrough respectively.This paper also makes the following contributions:(1)The memory replacement strategy of distributed shared memory is analyzed,and an effective strategy is proposed along with the implementation method in virtualization environments.(2)A distributed shared memory is implemented in the QEMU-KVM system as part of the distributed virtual machine monitor dQEMU.This paper elaborates some specific problems and their solutions in the implementation,revealing the key points of implementing distributed shared memory in virtualization environments,and thus has certain reference value for future work.(3)In the implementation of distributed shared memory,both TCP and RDMA network communications are supported,and the RDMA network is used to improve the performance of distributed shared memory.This paper tests dQEMU,equipped with the aforementioned distributed shared memory,on a cluster,and verifies that mainstream operating systems such as Linux can run on virtual machines started by it.Experiments on memory synchronization delay and memory read/write throughput in different scenarios show that using RDMA network communication in distributed shared memory can achieve lower latency and higher throughput.Distributed shared memory also exhibits different performance characteristics when running different types of applications,with computationally intensive applications performing best.