Research And Implementation Of Container Scheduling For Big Data Services

The development of media technologies such as cloud computing,the Internet of Things,and social networks has led to an explosion of all kinds of data around the world,and the demand for services that the Internet needs to handle has also grown rapidly.Therefore,during the evolution of the Internet,massive amounts of data processing and service requirements became a major challenge.At present,big data services are widely used for the business processing of massive data.Big data services are often deployed in clusters on traditional physical or virtual machines,which often weakens the performance and flexibility of the services.Leveraging the advantages of an emerging lightweight virtual technology container in terms of performance and flexibility,deploying big data services into the environment built by container technology can guarantee the high performance and flexible life cycle management of service runtimes.In big data application scenarios,the service requirements of many businesses are unpredictable.When faced with sudden changes in data load,it is important to reasonably schedule dynamic services.Building a containerized big data service and performing container scheduling based on service performance can meet high availability and high performance requirements at a lower cost,ensuring the normal operation of services and host nodes.Through the above explanation and analysis,the main research work of this paper is as follows:1.In order to solve the single-granularity level big data service monitoring problem of the existing monitoring architecture,from the perspective of global multi-granularity level monitoring,a container monitoring architecture for big data services is presented.The monitoring architecture mainly includes two parts: monitoring service and alarm response service.The monitoring service is responsible for monitoring the multi-granularity level metric data in the big data service,and integrates a monitoring display platform to grasp the changes of monitoring resources in real time.The alarm service has completed the design of alarm rules and alarm response mechanisms,and realized the management of alarm information and the response processing of alarms.2.To improve the lack of research on service level scheduling,a container scheduling method for big data services is proposed.In order to solve the problems in service scheduling,this method analyzes the needs of big data service container scheduling,and clarifies the scheduling objects and scheduling processes in big data services.In order to solve the static scheduling problem in the existing scheduling methods,the container scheduling method is customized and the implementation of core algorithms is given,which can meet the dynamic service requirements in the business processing process.3.Combined with traffic big data application cases,the feasibility and effectiveness of container scheduling methods were tested and analyzed,and the main performance metrics of services before and after container scheduling were compared and analyzed.Experiments verify the feasibility and effectiveness of the method in this paper.By performing container scheduling for big data services,service performance can be improved and stable operation of services can be guaranteed.