Algorithm And System For Distributed Deep Ensemble Learning And Architecture Search

In recent years,deep learning algorithms represented by deep neural networks have achieved great success.However,deep learning still has problems such as large training samples,high computational resource requirements,and difficulty in tuning hyperparameters.To this end,researchers began to explore deep ensemble learning algorithms represented by deep forests.However,the existing deep forest training algorithms are single-machine and serial,with low training efficiency and high training time overhead.On the other hand,existing deep ensemble learning models need to rely on manual set.The modeling process has a high technical threshold and is time consuming and labor intensive.To this end,it is necessary to study the problem of automated deep ensemble learning modeling and structure search.However,the deep ensemble learning structure has a large search space and is complex,which makes the automated search of the deep ensemble learning model particularly difficult.Deep ensemble learning delivers outstanding performance across a wide range of tasks.However,in some resource-constrained scenarios,it is necessary to reduce the depth of deep ensemble learning.At the same time,resource constraints also impose stricter requirements on the efficiency of architecture search methods.Aiming at the above problems,this paper studies distributed parallel forest training algorithms and systems based on task parallelism,and distributed deep ensemble learning structure search methods and systems based on evolutionary algorithms.The main research work and contributions of this paper include:(1)The paper proposes a fine-grained distributed deep forest training algorithm based on task parallelism.Through efficient sub-forest uniform split-merge strategy and system layer optimization,an efficient parallel training method and distributed deep forest system are realized.The experimental results show that the system can achieve an order of magnitude acceleration compared to the existing deep forest training system gcForest.(2)The research implements a deep ensemble learning structure search method and system based on evolutionary algorithm.On the basis of defining and optimizing the deep ensemble learning structure search space,the research realizes a evolutionary algorithm based structure search algorithm,a generalized search algorithm framework of deep ensemble learning model and realizes an efficient distributed deep ensemble learning structure search system.The experimental results show that the proposed method can obtain a deep ensemble structure superior to deep forests,and achieve significant improvement in learning prediction performance.(3)The study explores a dual-layer composite model of deep ensemble learning in a computing time-limited scenario.Based on the definition and optimization of the composite model search space,a two-stage composite model search method is proposed,which includes the model rapid screening stage and the model selection stage.The experimental results show that the automated search of the dual-layer composite model is better than the learning prediction performance of Auto-sklearn,and has been deployed in a well-known large IT company.