Improving Datacenter's Resource Efficiency With Fine-grained Control Over Cloud Service Components

Cloud service providers improve the resource utilization through collocating latencycritical(LC)workloads with besteffort(BE)tasks in datacenters.However,they usually have to be conservative in resource allocation for BE tasks,for strictly guaranteeing the tail latency of LC workloads.In this paper,we show the inconsistent interferencetolerance feature of LC components,and prove that conservative approach will hurt the resource utilization at the components that contribute little to overall tail latency.We present Hebe,an aggressive controller that maximizes the resource utilization while guaranteeing LC service's tail latency requirement.Hebe identifies the service call paths of requests using a request tracer,and characterizes their durations on each component.Then,it analyzes the contribution of each LC component on the overall tail latency,and designs a thresholding mechanism to enable aggressive launchment of BE tasks at components whose contributions are small.We evaluate Hebe using typical LC workloads with entirely different architecture and batch workloads.Compared with the previous work,we find that Hebe can improve the system throughput by 7%-18%,CPU utilization by 10%-27%,and memory bandwidth utilization by 13%-25%,while guaranteeing the SLA(Service Level Agreement).