Research On Deep Sharing Ensemble Networks

Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of ion.These methods have dramatically improved the state-of-the-art in pattern recognition,speech recognition,visual object recognition,natural language processing and many other high dimension domains.However,due to the large number of layers and large parameter scales,deep learning often results in gradient vanishing,falling into local optimal solution,over-fitting and so on.Deep learning models always need lots of training data,skilled training skills and lots of time to select appropriate hyper-parameters.Ensemble learning methods could combination many individual models into a better supervised model.Traditional ensemble learning models have good performance,but they are considered to be shallow learning model and how to build deep model is still a hard problem.In this paper,we use ensemble learning methods to propose a novel deep sharing ensemble network,this methods through joint training many independent output layers in each hidden layer and injecting gradients can reduce the gradient vanishing phenomenon and get a better generalization performance.Compared to the traditional ensemble learning algorithm,the deep ensemble networks can greatly reduce the number of parameters and save the training time by sharing the hidden layer.This method can also easily extend to be deep shared convolution ensemble networks and deep sharing recurrent ensemble networks.Deep sharing ensemble methods can generally improve the generalization performance of meta model 1%?3%.