Design And Implementation Of High Performance Communication Library Based On RDMA

With the coming era of big data, the development of Ethernet bandwidth from Gigabit to ultra-Gigabit as well as the problem of solving high-speed data processing by computer clusters has become a popular research direction. However, the problem of building a communication infrastructure between a computer clusters has gradually become one of the bottlenecks of the development of computer center. With high-speed parallel processing of GPU, cluster processing speed has been greatly improved. Thus, the real limit is the communication between computer clusters. RDMA was made to get the maximize use of network bandwidth over the traditional TCP / IP network transmission. The technology of high performance communication and GPU acceleration technology helps the computer clusters to perform best in the future of ultra-high-speed network era.In this paper, starts from the principle of high-performance communications, analysis the newly architecture of the TCP / IP and the implementation of high performance communications over Infiniband. By analyzing the TOE technology as well as the principles of RDMA technology, comparing to the socket solutions and Accelio, and some architectural advantages, there is a solution of how to highly perform over RDMA, which supports for GPUDirect RDMA technology.This paper describes the design and implementation of communication libraries in details. The core of the communication libraries is the using of Buddy Partnership memory management algorithms to solve the registration delay of RDMA by pre-registration memory; With the use of pipelining technology can greatly improve the performance of GPUDirect RDMA, copy to GPU and send it to the remote by RDMA. Been tested, communication libraries achieve the target of maximum bandwidth utilization.