Research On Key Technologies Of Service Provisioning And Load Balancing For Microservices

In the era of Internet of Things(Io T),microservices are emerging and are expected to be the next generation of service architectures.Microservices bring several advantages,such as improving reusability,enabling agile software development,and enhancing scalability.Leading microservice solutions will become the technology-driven force for the Io T industry,and the key support for the national Io T infrastructure,driving the Io T service industry evolution.However,there are three critical issues that remain unsolved in existing microservice solutions.Firstly,since different components of applications are dependent on each other,it is hard to provision dynamic microservices via monolithic applications.Secondly,balancing traffic loads across microservice instances burns host CPUs,taking away from the CPU cycles available to business logic.Moreover,traditional hierarchical request scheduling schemes suffer from load imbalance,dispatching redundantly,and stealing tasks frequently,which causes unpredictable Qualityof-Service(Qo S)for microservices that serve in a chained fashion.This thesis takes efficient microservice provisioning and load balancing methods as the study objective.The main work and contributions are as follows:(1)To solve the issue of dynamic microservices provisioning,a microservice provisioning scheme based on transparent computing from the perspective of loading service on-demand,named BOAT,is proposed.BOAT disaggregates application installation packages into numerous microservices on-demand at runtime,in the unit of code and resource blocks.Such that,Io T devices only need to load the necessary blocks of an application to obtain the requested services.BOAT designs a fine-grained relocation abstract to support symbol relinking within code blocks and enable the execution in a block-streaming way.Moreover,BOAT develops a lightweight file system to support the on-demand loading of resource blocks.We implement the proposed BOAT scheme on wearable devices.The experimental results demonstrate that BOAT can effectively improve the scalability of lightweight Io T devices,as well as reduce the service loading delay.(2)To solve the issue of load balancing burning host CPUs,a performance optimization scheme of offloading software load balancers onto multi-core Smart NICs,named NICLBs,is proposed.After surveying representative commercial multicore Smart NICs,we summarize the features of the ideal Smart NICs suitable for offloading load balancers,based on the requirements of load balancing scenarios.Then,we perform detailed measurements to profile the Smart NICs’ performance characterizations.It is observed that compared with general servers,the Smart NICs have wimpy synchronization and memory access capabilities.Further,we design and implement the NICLBs based on the studied characterizations.NICLBs builds a decentralized architecture with multi-queue,multi-core parallelism,and stateless properties,to exploit the many-core of the Smart NICs while avoiding expensive synchronization overhead.NICLBs also employs an agent-based allocation mechanism for global resource sharing,to eliminate synchronization bottlenecks caused by shared scheduling states.Moreover,NICLBs exploits bulk execution and data prefetch mechanisms to hide memory access latency.Experimental results show that NICLBs can build efficient load balancers on cheap multicore Smart NICs.(3)To solve the issue of unpredictable Qo S incurred by hierarchical scheduling,a performance optimization scheme of core-granularity load balancing based on Smart NICs from the perspective of vertical integration,named RPCDirect,is proposed.By taking advantage of the Smart NICs staying close to the network,RPCDirect directly schedules RPC requests to worker cores on Smart NIC-enabled load balancers,thus eliminating the latency and resource overheads introduced by the second scheduling,avoiding load imbalance between servers,and straightening the curved packet paths.Firstly,we prove that the proposed core-granularity scheme brings better scheduling quality than the hierarchical ones,based on queuing theory.Then,we employ a core-granularity request scheduling abstract of transforming the dispatching of remote physical cores into the dispatching of local virtual workers,which achieves efficient scheduling while guaranteeing scheduling quality.Finally,we design an abstract of steering packets to specific cache slices,based on the bimodal access characteristics of the last level cache,which enables preloading requests into the slices close to the target cores.Experimental results show that RPCDirect can significantly improve the 99 th percentile tail latencies.There are 61 figures,7 tables,and 223 citations in this thesis.