Research And Implementation Of RPC Framework Based On Multi-Path RDMA

In the field of data processing system,efficient end-to-end data communication and remote data access is a very challenging problem,in which the quality of network performance directly affects the quality of communication and service.To speed up end-to-end communication,improve communication efficiency and reliability,Remote Direct Memory Access is used.RDMA integrates the packet,unpacket and data verification of the network data stack into the RDMA network card,which is realized by hardware and liberates CPU computing power.In addition,RDMA technology also achieves data transmission without the participation of remote CPU by caching page tables inside the network card.Due to its high bandwidth,high throughput and low communication delay,RDMA has been adopted by more and more data processing centers,especially for data intensive applications.However,the widely used RDMA technology does not take full advantage of the data center multiplexing network.Due to the communication characteristics of RDMA network,most of the end-to-end RDMA network connections are designed for a single link,which does not take advantage of the hierarchical topology of data center network and ignores the characteristics of end-to-end multi-link.Correspondingly,the RPC framework based on single-link RDMA also ignores the performance advantages brought by multi-link.In addition the design of RPC based on multi-link RDMA is extremely challenging.The primary reason is the hardware constraint.The design scenario of RDMA network adapter is a full-link RDMA network,and the traditional Ethernet-based RDMA network such as Ro CE is not considered,so the multi-channel feature is naturally excluded.The second is the complexity of RPC in multi-way RDMA network,so the link selection,out-of-order arrival and other problems brought by multi-way network should be fully considered.In view of the above problems,this paper makes the following work:1.This paper designs and implements a multi-channel RDMA network,which gives full play to the hierarchical topology of the underlying network End-to-end multilink feature.A single data flow is divided into multiple sub-data flows based on link strengths and weaknesses Multichannel RDMA network for data transmission.In addition,multiple RDMA networks can be used for bursty networks.It responds quickly to route anomalies and network congestion,and dynamically schedules subdata flows to avoid network fluctuations and Performance degradation caused by congestion.2.This paper designs and implements a user-mode RPC framework for multi-channel RDMA network,which makes full use of the characteristics of multi-channel RDMA and realizes the characteristics of multi-connection,load balancing and dynamic connection.The control surface RPC and data surface RPC are processed separately,and different RDMA communication primitives are used.Compared with the traditional RPC framework,the RPC framework implemented in this paper has the advantages of higher throughput,dynamic scaling and low latency.