Based On Active And Passive Mode Distributed Proxy Cache Server Design And Implementation

Rapid expansion of web applications will definitely increase the network load. According to statistics, more than two third of the network congestion is caused by web. How to mitigate the network payload and get a better experience of internet? These questions have got more and more concerns by internet users. However, traditional proxy is usually functional simple and large amount of web cache is far exceeded its ability.Based on the newest HTTP 1.1 protocol and new web crawler technique, the design and implementation of a new distributed proxy cache server with both active and passive mode will be introduced in this thesis. This server used a lot of new feature of Linux kernel 2.6, such as EPOLL asynchronous event driven policy, which optimizes the network I/O. Meanwhile, to reduce the separate malloc and free of raw memory, a memory pool based on objects, which is according to the kernel memory allocate policy Slab on Solaris, was implemented. It can sharply reduce the memory malloc and free. Also, to decrease the influence of network attack to cache files and to make load balance, this system take an architecture with a front redirector and several background cache server. When system's payload is under the low water-level for a certain time, it will start the active mode, which will update and get web page based on a continuous refreshed list and user's strategy configuration. In addition, a timer queue is designed to meet the need of timer, such as connection timeout, file expires. Also, the system keeps a DNS cache, which will flush to disk regularly, to avoid the cache lost during system down and reboot. Last, the system supply with kinds of logs.The system is developmented on Suse 11 server edition. By actual environment and simulation test of the system's function and performance, it can serve a large amount of users with high efficiency.