A High Performance Network Content Reconstruction System

With the development of network technology and the enrichment of network applications, network content security is becoming increasingly prominent. As information leakage and illegal information dissemination lead to serious consequences, the strengthening of network content supervision is an urgent problem which needs solution, and network content restoration is the core of network content supervision.Network development brings new challenges to the network content restoration. Firstly, with increasing network link rate, the network content restoration of numerous concurrent flows needs to be supported, and as the process of content restoration consumes mass memory, there is an urgent need for efficient cache management. Secondly, the supervision of network content must be implemented in real time, which demands higher speed of the content restoration. Thirdly, in order to speed up the content transfer speed, several application protocols divide the content into blocks and transmit the blocks in parallel, so the traditional content restoration techniques which aim at single connection can not solve the problem.Aiming at solving the cache management of content restoration systems, a size-perceptive cache allocating method is proposed in the paper. For cache allocation, the size distribution of the contents transmitted in the network is measured, and the characteristics of content distribution are taken as the basis of cache allocation. The cache allocation makes a balance between memory waste and the repeated memory allocation which is caused by cache increase, and minimizing memory waste while occupying less system resources. Experiments show that the present cache allocation effectively reduces the waste of system resources.For content flow ageing, a hybrid flow ageing method is further proposed, which is based on TCP connection flags and measures timeout feature of the network content flows. In the method, the timeout value of content flows which receive a bidirectional FIN-marked packet or a unidirectional RST-marked packet is set to 5s, and the timeout value of other content flows is set to 64s. By assessing the error confidence interval, the efficiency and the feasibility of the hybrid flow ageing method are proved in theory. Experimental results show that compared with the fixed 64s timeout strategy, the real-time property of the flow ageing method is increased, and the occupation of system memory resources is reduced.To address the problems including content flow reconstruction, multi-connection transmission and so on, an extensible network content restoration system which supports multiple connections is designed and implemented. A bidirectional content flow table is designed, and an appropriate hash algorithm is adopted for realizing fast TCP flow reconstruction and restoration of associated multi-connection contents. Based on the characteristics of object-oriented programming languages, the protocol scalability of the content restoration framework is achieved. The content restoration system makes best use of the advantages of a multi-core system, and speeds up the content restoration with multi-thread processing. For dealing with traffic bursts in network, the buffer queue sharing of data packets is introduced, thus avoiding influence caused by packet loss. Finally, the content restoration system is implemented in the paper, and the functional and performance testing of the system is executed in actual links.