Research On Distributed Caching Technology Of Network Traffic Based On RDMA

At present,as network security is becoming more and more severe and attack means are becoming more and more diversified,a more refined analysis of network traffic is required.However,with the continuous development of the Internet,network bandwidth is gradually increasing,from the original 10 Gbps to 100 Gbps and even reaching up to 200 Gbps.Thus a refined analysis of network traffic will bring huge performance pressure to network traffic analysis tools and related applications.At this time,it can yet be regarded as a solution to use distributed systems with high throughput and concurrency to perform distributed caching of network traffic,and then conduct a multi-task,parallel and refined analysis and detection of network traffic.However,at present many distributed caching systems in industry have the following problems in the caching of network traffic: First,most distributed caching platforms have no specificity and target general data types,such as strings,lists,and others.And they are unable to meet the requirements of the applications of network traffic,such as TCP stream query;Second,these distributed caching platforms basically use TCP/IP protocol for communication and data transmission between nodes,which cannot meet the high-performance requirements for data transmission in large-scale network traffic scenarios.In response to the above problems,the following work has been carried out in this thesis:In response to the first problem,a distributed caching platform specifically for network traffic is designed and implemented,and related services specifically for network traffic applications are provided,including traffic caching,traffic acquisition,and TCP stream query.In terms of the second problem,as experiments show that the performance of RDMA network communication and data transmission is much higher than that of TCP/IP,therefore the new network communication technology RDMA and Mellanox network card hardware are used to improve the performance of network communication and data transmission between nodes in a distributed system.The entire platform mainly includes the design and implementation of the Master module,the Slave module,and the RDMA asynchronous communication module.In order to ensure the normal interaction between the nodes in the distributed platform,the communication protocol between the nodes is designed in detail.After the completion of the development of the entire platform the throughput and concurrent query performance of the platform have been tested in multiple dimensions by constructing different types of network data packets.The test results show that the six slave nodes can support traffic caching and acquisition of nearly 200 Gbps as well as,concurrent query performance of 100,000 traffic per second.