Research And Design Of High-performance Network Traffic Control System

The development of Internet technology made the network become a new carrier of medium, and the rapid growth of the medium has brought a great impact to the Internet vice versa. Especially the infiltration of video, personal medium, traditional television and other forms of medium into the Internet made the network traffic increased dramatically, which cause the operating and managing cost of ISP grow rapidly. ISP has used some methods of traffic-limiting to control the network traffic, but these methods limited the development of media also, and were not benefit to the further development of the Internet. So inventing of a new technology to control network traffic has become a focus of research.At present, the Internet traffic was mainly generated by P2P and HTTP. It was estimated that these two kinds of traffic have accounted for more than 70% of total traffic, and still showed a rising trend. So it is important to control the traffic generated by P2P and HTTP. When reducing the traffic generated by P2P and HTTP the overall network traffic will be effectively reduced. The traffic control system described by this thesis was specific to P2P and HTTP. This system was deployed at the outlet of network to cache the P2P and HTTP traffic. By responding the following requests for the same resource with the cached resource, the network traffic and bandwidth will be reduced, and the user experience will be improved.This system was based on the Linux platform, and used many design patterns in the process of design. The high-performance was the key demand of the system, because the system was deployed at the outlet of network, which would face heavy stress of service. Several measures were adopted to improve the performance. First was the concurrency. Because the system would receive a large number of requests in a short time, the traditional model of one connection per thread could not cope with such a high concurrency situation. For this reason, the pattern of reactor which based on Linux2.6's epoll system call was adopted, so that the degree of concurrent was only subject to the number of file descriptor(theoretical upper limit was 65536) in the operating system. Second was the disk I/O. The disk I/O was a hot spot in the system. To improve the capacity of throughput of the disk the pattern of thread pool was used. The complexity of implementation was reduced because this pattern separated the disk I/O from communication mechanisms; and the capacity of throughput of the disk was improved because there were many threads in the thread pool dealing with the disk I/O simultaneously. Finally was the organizing and indexing of the data. The data was stored in the form of files which were organized into three level directories. In order to speed up the speed of retrieval, the system constructed an index in the memory for all the files. Through this index a file could be located quickly.This system has been deployed widely. The data obtained from a practical use show that the system had strong concurrency, high disk throughput and could reduce network traffic effectively.