A Stochastic Model For Analyzing Tail Latency Of Online Cloud Services

Recent years have witnessed the rapidly growth of cloud computing market.A massive amount of enterprise applications,like social networking,e-commerce,video streaming,are moving to cloud systems.Resource sharing,multi-tenant interference and bursty workloads in cloud computing lead to high tail-latency that severely affects user quality of experience(QoE).And the prediction of tail latency has become a hot topic in current research.At present,there are some research on modeling analysis of tail latency of services.However,there still lacks an efficient and easy-to-use analytical model to analyze the tail latency of response time of online cloud services.A big challenge is that online cloud service typically consists of multiple tiers of components that interacts with each other,which is difficult to be accurately modeled.An effective stochastic model to analyze the tail latency of online cloud services was proposed in this paper.First,based on queuing theory,Stochastic reward net(SRN)is used to model single-tier servers.And then,as online service is usually a multi-tier architecture,the paper uses the single-tier architecture to model multi-tier online cloud service according to the processing of requests at the server side by taking the interaction between multi-tiers into account.Based on the multi-tier architecture model,some performance indicators of the system under steady state,such as availability,average response time and so on,can be calculated by using Stochastic Petri Net Package(SPNP).Secondly,based on the multi-tier architecture model,a tagged customer model is then introduced into the SRN to compute the cumulative distribution of response time,based on which tail latency can be directly derived.an e-commerce site was further implemented on both a private cloud and a public cloud to verify the accuracy and effectiveness of the proposed model,respectively.A series of experiments show that the differences between the tail latency obtained from the model and that from the actual experiments are generally less than 20% in both cases.