Optimization Of Asynchronous Graph Processing System Based On RDMA's Asymmetry

Many problems in the real world can be modeled as graph problems,and the scale of graph data is in a constant state of growth.In order to effectively analyze and compute large-scale graph data,distributed graph processing systems were developed.The classical distributed graph processing system usually uses traditional TCP/IP communication.However,in the real graph scenario with power-law distribution,the average number of vertex copies of the asynchronous graph processing system based on vertex-cut increases rapidly.This,coupled with the slow speed of TCP/IP communication,has resulted in a significant increase in communication overhead between master and slavers,which becomes a bottleneck in system overhead.Frequent communication between master and slavers is a major cause of excessive system communication overhead.Remote Direct Memory Access(RDMA)is characterized by low latency,high bandwidth,and low CPU overhead.Therefore,replacing TCP/IP communication with RDMA communication will result in a small amount of acceleration.Moreover,the communication structure between the master and slavers of the power-law graph exhibits the one-to-many property.Using this structure,an efficient communication mechanism based on RDMA's asymmetry(i.e.,the overhead of out-bound RDMA is much higher than that of in-bound RDMA in the one-to-many scenario)is designed.Under this mechanism,in-bound RDMA and out-bound RDMA can be distributed separately on the different two parts of machines in order to make better use of RDMA's asymmetry,which makes communication overhead reduced.In order to evaluate the proposed optimization scheme,it is implemented on the classic asynchronous graph processing system Power Graph and tested in large-scale graph datasets.The experiment results show that the optimized system based on RDMA's asymmetry has a reduced runtime of about 35% compared to the Power Graph by running the Page Rank algorithm on different types of graphs.