Research Of Extreme Learning Machine Based On Ensemble Method

Extreme learning machine(ELM)is a learning algorithm based on single hidden layer feedforward network.The weights and biases between the input layer and the hidden layer are determined by random methods,and the weights of the output layer can be further determined by the analysis method.ELM overcomes many shortcomings existing in the gradient network algorithm,such as falling into local extremum,inappropriate learning rate,slow learning speed,etc.,but ELM also has the hidden danger of overfitting and the stability of a single algorithm is relatively poor.The research work of this paper is as follows:In order to solve the above problems,this paper proposes an ensemble model of diversity regularized extreme learning machine(DRELM).The basic idea of DRELM is to randomly select the input weights for each ELM by changing the distribution of hidden layer node parameters.The Leave One Out cross validation method is used to find the optimal number of hidden nodes for each base learner,calculate and output the optimal hidden layer output weight,and train better and differentiated base learners.Then,the new penalty terms related to diversity are added to the whole network objective function,and the output weights of the hidden layer of each base learner are iteratively updated.Finally,the final output results of the whole network model are obtained by integrating the output results of all base learners and averaging them.This method can effectively realize the fusion of regularized extreme learning machine(RELM),taking into account accuracy and diversity.The experimental results on 10 UCI datasets with different scales show that the proposed research method is effective.In general,the hidden layer parameters of the extreme learning machine are randomly selected,which inevitably leads to low utilization efficiency of some hidden layer nodes,and samples close to the classification boundary may be misclassified.In order to solve this kind of problem and improve the training speed and stability of the model at the same time,we propose a new model,called the ensemble method of diversity regularization extreme learning machine(C-DRELM)based on constraint and feature selection.This method is improved based on DRELM algorithm.The weight and bias between input layer and hidden layer are calculated by using the difference of samples between different classes to form the difference vector set,which replaces the previous random selection method.Then,the redundant information is removed by the embedded feature selection method,which improves the training speed of the model.Finally,the network output weights of each base learner are iteratively updated according to the objective optimization function with diversity and regularization terms.The experimental results show that C-DRELM algorithm has higher performance and robustness than ELM,C-ELM and DRELM algorithms.