Research On Hierarchical Gradient Boosting ELM Algorithm

In recent years,neural networks based on gradient descent method have made great achievement,many new in-depth learning algorithms have emerged.Although it has achieved great success in the industry,training such a network will cost a lot of time.There are many algorithms do not need to use a gradient in the field of machine learning,extreme learning machine(ELM)proposed by professor Huang Guangbin is one of them.It is a single hidden layer feedforward neural network,the weight of the hidden layer is randomly generated,the calculation of output layer is based on the minimum mean square error of generalized inverse.Because ELM does not require iterations,it computes very quickly.However,due to its shallow architecture,feature learning using ELM may not be effective,even with a large number of hidden nodes.To address this issue,in this paper,a new ELM-based Hierarchical Gradient Boosting Learning Algorithm(HGB-ELM)is proposed.The proposed algorithm is divided into two main components.An ELM-based unsupervised autoencoder network followed by supervised learning to select the features of data.In this part,the random nodes of any hidden layer are inactivated.At the same time,a strategy of increasing variance in turn is used to initialize the weights,and simple weighted average ensemble method is used to improve the stability of the extracted features.The training phase is calculated in two steps,the first time a sample is used to calculate to reduce the risk of over-fitting.The other part of the method is to improve the generalization ability of the network by the method of the gradient boosting,so that it can get higher precision.A large number of experimental results show that this new algorithm HGB-ELM has greater generalization capability,and the structure of the multilayer also has wider scope of application.