| With the development of high-performance computer systems,the complexity and computing performance of a single computing node is increasing dramatically.In order to provide a well-balanced bandwidth in terms of bytes-to-flops ratio,Multi-Rail net-works become the development trend for high-performance interconnection networks,and have been used by many leading HPC systems.In Multi-Rail network,each com-puting node is connected to the network through multiple ports(rails)to get ultra-high bandwidth.However,how to effectively arrange the multiple links of computing node to access the network and make full use of network capacity still suffers from a lack of in-depth research.The goal of this paper is to fully understand how different computing node connection modes affect the performance of Multi-Rail networks.Considering the wide application of Fat-tree,this paper focuses on the influence of different node connection modes on Multi-Rail Fat-tree network.Multi-Rail Fat-tree network can be divided into Single-Plane Multi-Rail Fat-tree network and Multi-Plane Multi-Rail Fat-tree network,according to the number of sets of network devices.The main contributions are:(1)The connection modes of computing nodes in Single-Plane Multi-Rail Fat-tree network are analyzed.According to the structure characteristics of Fat-tree,a kind of topology called MR-tree is proposed to describe all the balanced and encapsulation friendly connection modes of computing nodes in Single-Plane Multi-Rail Fat-tree network.The deployment complexity of different configurations of MR-tree is analyzed.The analysis shows that the actual cost of different node connection modes is roughly the same as that of traditional mode of parallel links through reasonable physical encapsulation.Through theoretical analysis and simulation,the differences of MR-trees with different parame-ters in terms of average path length,path diversity,fault-tolerance,network latency and throughput are discussed.The results show that if each node connects to more pods,the average packet latency will decrease.Meanwhile,parameters of MR-tree and network oversubscription ratio have a comprehensive effect on saturation throughput.(2)The connection modes of computing nodes in Multi-Plane Multi-Rail Fat-tree net-work are analyzed.On the basis of MR-tree,a new kind of computing node connection mode called MP-tree is proposed,which can reduce the average shortest path length of Multi-Plane Multi-Rail Fat-tree network.The design idea of MP-tree is that each plane adopts MR-tree with identical parameters,but the nodes that need to communicate across the core layer should be different in each plane as far as possible,so as to reduce the aver-age shortest path length of the whole network.According to the structural characteristics of MP-tree,three routing algorithms are proposed,and other characteristics are also ana-lyzed.Simulation results show that MP-tree performs better than mirrored mode in terms of latency and throughput.In terms of different routing algorithms for MP-tree,when the Fat-tree switching network has full bisection bandwidth or a little oversubscription,the ANCA routing with offset constant performs best.In the case of a large number of oversubscription,the minimal routing will have better performance.The research in this paper fully demonstrates that for Multi-Rail Fat-tree network,reasonable computing node connection mode can improve the performance of the net-work.Therefore,the connection mode of computing nodes should be fully considered in the design process of high-performance interconnection networks.This paper develops the classical topology of Fat-tree and fills the gap of Fat-tree structure under the back-ground of Multi-Rail network,having moderate practical significance for the development of HPC systems. |
