The Cluster Design Of The Web Proxy Server

As the sub-system of "Development of supporting system of IPV6 Stream distribution application system", sponsored by research fund of the "IPV6 based next generation internet key technology and industrialization promotion" of Beijing Municipal Science and Technology Commission, the new type of cluster based HTTP proxy servers is applied in real-life task. Increasing applications of WEB service give rise to contradiction between the limited bandwidth and user experience. Therefore, ISP (internet service provider) on the one hand have to solve the problem, on the other hand have to satisfy their subscriber, thus comes the age of cluster-based HTTP proxy servers. By settling transparent cluster-based HTTP proxy servers at the outlet of LAN, nearly 8% cost are saved.This thesis proposes a new type of transparent cluster-based proxy servers. Transparent proxy, is commonly known as TCP hijacking proxyer, in common with Network Address Translation (NAT) in some aspects. Transparence makes its users not aware of it, thus frees its users from the hassle of setting up individual browser and avoids trying to convince users to type the diameters they may not understand at all. Besides, it may help ISP save its cost as well. Transparence means saving the time and cost for both users and ISP, while clustering improves the I/O performance of the servers, shortens response time.At the beginning of the thesis, series of terminologies and protocols are introduced which are related to the architecture of generic proxy server and virtual cluster server. The balance algorithm of Round-Robin based on IP applied in Load Balance Engine then is also sujested. To alleviate the pressure on Front Filter which intercepts thousands of TCP packets per second, epoll-driven mechanism in LBE is applied. By analyzing different network servers' architectures and several event-dispatch mechanisms, a high performance and scalable cluster-based HTTP server which follows open standard is outlined. It's developed on the base of Linux OS and powered by asynchronous event driven framework as featured in Linux 2.6 kernel in which it provides a uniform interface for registering event and handling the asynchronous event callback. To design and achieve high performance, hash function, mempool and thread pool are also used to speed up the searching and transmitting files on disk. The rest of thesis introduces the details of both design and implementation. The results of tests show the designs have obtained expected scalability and I/O performance. Also some drawbacks of its implementation are mentioned in the last chapter.