Research And Implementation Of A Cluster-Based Cooperative Web Caching System

With the rapid expansion of Internet, the improvement of network bandwidth cannot catch up with the increase of network users. Web proxy caching technique provides a solution to this conflicting situation. As it can help to reduce the Web latency and traffic effectively, and alleviate the workload of Web servers, Web caching is now widely applied on Internet. However, with the rapid increase in Web clients all over the world, the performance of single cache can hardly satisfy them. Especially at the exit of large ISP, servers of large ICP and upper nodes in hierarchical caching system, high performance caching servers are needed. These requirements accelerate the development of cluster-based caching technique. Cluster-based caching technique takes full advantage of cluster technology, and introduces cooperative mechanism among the proxies in cluster, thus organizes these self-governed proxy nodes into a whole caching system. So it can improve the performance of single caching server, reduce Web latency and traffic farther.By introducing effective cooperative mechanism, the cluster-based caching technique can evidently enhance hit rate, decrease users' waiting time and network taffic. Many cooperative mechanisms have been advanced now. ICP is the most successful amony them, so it is almost used in every cluster-based caching system. But it is too complex for the cluster environment, and damages the scalability of cluster. Aiming to overcome these disadvantages, we introduce the digest caching communication mechanism for proxies' cooperation, and implement a cluster-based cooperative caching system.Firstly, we describe the design of a proxy caching server. The functions that weachieve in this server are serving for multiple clients at the same time, rapidlysearching documents in cache, documents replacement, checking consistency, etc.Then we propose the design of cooperative mechanism among proxies. We decide toadopt digest caching communication mechanism. This mechanism can help toimprove the scalability of our cluster caching system. At last, we describe how torealize a cluster-based cooperative Web caching system, and then use the WisconsinProxy Benchmark (WPB) to test the performance of our system. The results indicatethat our system can reduce interproxy communication overhead, while keepingalmost the same latency and caching hit rate as ICP system does.