High-speed Network Data Capture And Parallel Processing On Multi-core Platform

Information technology is advancing rapidly. The trends of multicore architecture of CPUs and parallel programming model are becoming more and more apparent. With the increasing of network bandwidth, the amount of network data flies into an explosive increase. The demand of quickly processing the network data in the advantage of multicore architectures is increasing with the emergence of the internet of things. While multicore is more popular, it becomes hot topic that how to multicore architecture strengths in software effectively. The paper proposes a solution of network data capture and data parallel processing on the multi-threaded multi-core platform. This subject can be applied to modern network monitor system.The subject belongs to the internet content security. After it makes a simple introduction of development in this field and related products, it briefly analyze the technical background and advantages and disadvantages of existing solutions of parallel processing and data capture part. The article elaborates on these two parts. First it analyzes the characteristics and requirements of packet capture technology, and its existing implementations. Then the paper designs the zero-copy packet capture platform by the research of network data packets mechanism through the NIC. The NIC transfers the packet to a buffer which allocated in user-space by DMA when a packet received. In this way, the packet capture platform can reduce the times of data copy and system calls, and also can improve efficiency of CPU usage.Then, it designs parallel processing of network data. The design follows the parallel model, taking the advantages of multi-core multi-threaded architecture. The bottleneck of the protocol stack in processing the packet is explored and summarized after the analysis of fundamental principle of the TCP/IP protocol stack. Serial protocol stack cannot adapt the multicore platform totally. Parallel protocol stack is designed in order to maximize hardware utilization and overall system performance.Finally, the paper explains the system architecture and interfaces in detail. After the analysis of the existing shortcomings of similar designs, data processing device is designed for the parallel protocol stack. Experiment and results of the system show that the coupling of zero-copy, data dispatcher device and parallel protocol stack effectively improve overall system performance, especially in multi-core processor platform.