Optimization Of Resource Management Throughout The Application Processing In Datacenter Networks

Cloud services are providing more and more convenient services for network users,while the resource heterogeneity and different server configurations bring serious chal-lenges to the network performance.These challenges lie in the overall procedures of ap-plication processing.First,when an end user sends a request to one network application,the server should make access control and decide whether to accept it or not.Then the server should further schedule all the accepted requests and make transmission control.When this request is being served,it will call for server communications and coprocess-ing to satisfy the function requirements.For the request that needs cross-region services,different IDCs(Internet Datacenters)have to make information synchronization and da-ta transmission.At last,the final calculation results could be sent back to this request.To provide high performance and shorter latency to network users,this paper clarifies the overall procedures of network application requests.For each procedure,this paper proposes a customized solution and optimization scheme.Overall,this paper designs a resource management solution for the network application processings,achieving fair ac-cess control,reasonable task scheduling,controlled server communication and efficient data synchronization.The contributions are summarized as follows:1.The access control procedure.This paper makes a measurement on online ser-vice workloads,finding the limitation of the current access control scheme,disclosing the crux of the performance gap among different applications.Then we propose a D~3G Algorithm(Dynamic Differentiated service with Delay-Guarantee),reducing the latency of interactive workloads and ensuring application fairness.2.The scheduling and transmission control procedure.This paper gives a survey on task serialization,analyzing the scheduling inefficiency of the state-of-the-art protocols.Then we design a cross layer scheduling scheme named TAFA(Task Aware and Flow Aware),which improves the scheduling results by taking both task and flow information into consideration.Furthermore,TAFA also modifies the transmission control protocol by adopting multiple priority queues.Experiments show that TAFA reduce 35%task completion time for datacenter applications.3.The server communication procedure.For services that need server communica-tion or coprocessing,the communication latency in network is unneglectable.This paper measures the real network of Baidu Company,pointing out the bottleneck of container-ized network functions.To achieve both low communication overhead for containers and high throughput for datacenter system,we design,implement,deploy and evaluate a flex-ible solution for container groups management,called FreeContainer,which is proved to be able to increase overall throughput by 90%at most on Baidu Network.4.The data synchronization procedure.To ensure service consistency and stabil-ity,the cross-IDC duplicate synchronization is the key procedure.This paper monitors the real network and disclose the disadvantages of the current data transmission scheme which is based on P2P.By proposing a traffic maximization algorithm,this paper designs a cross-DR large data transmission scheduling system called PieBridge.Deployment and experiments on real network show that PieBridge reduces the transmission time to about30%compared to existing solutions and obtains promising performance results.