| In online service systems, delay experienced by a user from the service request to the service completion is one of the most critical performance metrics. To improve user delay experience, in this thesis, we investigate a novel aspect of system design: proactive serving, where the system can predict future user request arrivals and allocate its capacity to serve these upcoming requests proactively. This approach is complementary to the conventional capacity boosting mechanism and is motivated by recent industrial and academic advances. In particular, we investigate the fundamentals of proactive serving from a queuing theory perspective.;First, most importantly, we show that under proactive serving the average user delay decreases exponentially (in the prediction window size) for a wide range of queuing models. Furthermore, the delay reduction is robust against prediction errors. We also show that both the variance of user delay and the tail of user delay decrease exponentially under proactive serving, which are also important user delay experience metrics.;We then show that proactive serving is more effective in decreasing user delay than capacity boosting in light workload regime. In particular, the average user delay decays inverse-proportionally in system capacity, but exponentially in the prediction window size in proactive serving.;Finally we demonstrate how to leverage proactive serving in system design from a optimization point of view, e.g., how many resources are dedicated to proactive serving. The results provide useful engineering insights to system designers.;Our trace-driven simulation results demonstrate the practical power of proactive serving: for example, under the YouTube data trace of 1000 different videos, the average user delay can be decreased by 50% when the system predicts 100 seconds ahead. Our results provide useful insights for proactive serving and justify its increasing applications in practical systems. |
