Design And Implementation Of Docker-based Elastic Scheduling System For Live Cloud Platform

With the rapid development of the Internet,the traditional computer architecture can no longer meet the demand for explosive growth of data processing.The emergence of cloud computing provides a new way to solve this problem,so enterprises can handle additional service loads by purchasing cloud computing resources.In the last few years,most cloud platforms use virtual machine as the underlying virtualization technology,but the extra operating system mounted on the virtual machine will bring performance loss to the cloud platform,in addition,it is difficult for virtual machine to make rapid scalability response to the service loads due to its slow startup speed.Docker container technology is(?)similar but more lightweight virtualization technology,and it is considered to be a better solution for application distribution and deployment on the cloud platform because of its small additional resource overhead and shorter startup and destruction time.As a new type of Internet application,live streaming is extremely unstable in computing requirements and may cause network peak at any time,which brings a huge challenge to the business and maintenance of the enterprise.Based on the above background,this paper presents an elastic scheduling system in live streaming environment.This system can dynamically expand or shrink the services according to the real-time workloads so that it can respond to the network peak more quickly and effectively,eventually achieve the purpose of improving the system's resource utilization and saving the costs of maintenance and management.This paper first studies the related technologies of the Docker container,including the basic principles of the container,the underlying implementation and its application scenarios,etc.Then the overall architecture of the elastic scheduling system is designed and divided into three parts:resource monitoring module,scaling decision module and resource scheduling module.For the resource monitoring module,this article uses cAdvisor as the data collector,InfluxDB as the monitoring database and Grafana as the data display module.For the scaling decision module,this work focuses on the workload characteristics in the live streaming scenarios,finally the threshold-based scaling algorithm is used for CPU loads and the predictive scaling algorithm is used for network traffic loads.For the resource scheduling module,this paper schedules the underlying container resources by using the filtering and scoring scheduling policies,it also uses etcd to synchronize the container information between the master and slave nodes.Then for building Docker container cluster,this paper solves the problem of network communication in Docker container by using Flannel to build an overlay network,similarly,this article solves the problem of file sharing and data persistence of Docker container by using GlusterFS to build a distributed file system.In order to test and verify the elastic scheduling system,a corresponding test scheme is designed for its scaling function according to the application scenario,and finally the effectiveness of the elastic scheduling system is verified in the last of this paper.