Research And Implementation Of Dynamic Distributed System Of Ensemble Learning In Cluster Environment

Ensemble learning is proposed as a way to improve the overall prediction accuracy of the model.However,the traditional ensemble learning still has shortcomings.One is the problem of low efficiency of model training,that is,in order to obtain more stable model performance,the training running time of a single model is increasing,and the time it takes to combine and infer multiple weak learners can not be ignored.This paper aims at designing a Dynamic Distributed System of Ensemble Learning in order to improve the performance and stability of traditional ensemble learning.Aiming at improving the operation efficiency and realizing the dynamic adaptation of data,a dynamic ensemble learning system based on Kubernetes cluster will be constructed from the problems of the deployment of distributed ensemble learning and the dynamic adjustment and selection of weak learners.1)ensemble learning distributed deployment: The paper design the distributed deployment mechanism to minimize the operation efficiency of the distributed integrated learning training.2)Dynamic adjustment of weak learners: The dynamic adjustment mechanism of weak learner is designed to optimize the performance of single weak learner and improve the integration performance as much as possible.3)System and deployment of Kubernetes: This paper adopts Kubernetes as a distributed architecture for carrying ensemble learning.The interaction of ensemble learning application deployment combined with Kubernetes is designed.Through testing on the system from the aspects of mechanism testing,pod deployment efficiency,weak learner model container performance,distributed integrated learning efficiency and system stability,it is shown that the system can have faster training efficiency and higher prediction accuracy than traditional integrated learning.