Research And Implementation Of Network Packet Capture System On 10 Gigabit Network

With the rapid development of Internet technology, the bandwidth of network transmission has improved greatly. The transmission media is changed from coaxial-cable to fiber and the transmission rate is improved form 10 megabit or 100 megabit to 10 gigabit. High-speed transmission rate facilitate people to access information more quickly and conveniently. In the same time, however, it brings a new challenge for the protection of network information.Both applications and protection software in the field of network security are depended on the techniques of network packet capturing and analyzing. Nowadays, high transmission rate of the network is the bottleneck for packet data processing, instead of the limitation of low transfer rate and media[1,2]. The traditional techniques of packet analysis, such as wireshark and tcpdump. Under the high transmission rate of network, the loss rate of packet for such system will increases greatly, which causes a drastic decline in the performance. The main issue about the defend the data security is how to capture and analyze the big data with high efficiency. For this issues, the paper designs a packet capturing and analysis system to match 10 gigabit network.This system does a lot of work on the efficiency of data fetching and preservation. In view of the vast amounts of network data fetching, this system use the Endace DAG network capture card produce by Endace company as its hardware device, and the Libpcap library function that is one of the open source library used on Linux and UNIX as its developing function. The fetched data will be stored in Linux system with an ext4 file system. This system use the Endace DAG card to guarantee the packets loss rate is zero, and then filter the packets by demand. This system also use multi-threads to improve parallel processing capabilities while decreasing perceived interactive time. This system designs a concise directory structure to speed up the data storage. Furthermore, we design a more efficacious index structure to find the fragments of the HTTP data out.With the test, this system achieves the goals of development on the functional and non-functional requirements. The system still has many deficiencies and need to be improved in the future work.