SWEB++: Distributed scheduling and software support for high performance WWW applications

WWW-based information service has grown enormously during the last few years, and major performance bottlenecks have been caused by WWW server and internet bandwidth inadequacies. Augmenting the server with multiprocessor support and shifting computation to client-site machines can substantially improve the system response time and for some applications, it may also reduce network bandwidth requirements. Taking full advantage of these capabilities requires sophisticated scheduling.; We first investigate algorithms for scheduling HTTP requests within a server cluster. Typical distributed scheduling techniques for HTTP servers either use a simple round-robin request distribution algorithm, or have only a single criteria such as CPU load. We propose novel multifaceted scheduling techniques that optimize the use of a multiprocessor server by predicting the demands of requests on I/O, processor, and network resources. We present SWEB, a software system implementing our techniques on a cluster of workstations. We provide a performance analysis under simplified assumptions for understanding the impact of system loads when using our scheduling strategies. Our experiments show substantial improvements by our techniques compared to traditional algorithms, and we observe a close correlation between our theoretical analysis and the achieved results.; We then extend our techniques to adaptively incorporate client resources. Due to a wide variation in client capabilities and connection network characteristics, the standard technique of partitioning the client-server workload at a fixed point is infeasible. We present a task model and scheduling technique for adaptive client-server computing in which the computation required by a user request is dynamically partitioned between the client and server by monitoring network, client and server resources. We demonstrate the use of this technique in digital library applications such as image and postscript document browsing. We also present SWEB++, a software system to support programmers desiring to use our scheduling algorithms. Experimentally, we achieve significantly faster response times through utilizing client resources, and demonstrate the validity of our theoretical analysis.