Research On Theory And Optimization Method Of Online Sequential Extreme Learning Machine

The traditional BP neural network learning algorithm has some problems such as long training time,over fitting training and easy to fall into local optimum.The ELM comes from SLFNs proposed by Huang in 2004,it has the advantages of simple structure,fast learning speed and good global optimization ability,which becomes a frontier direction in machine learning field.Subsequently,Liang proposed an online sequential ELM algorithm named OS-ELM.The algorithm solves the problem that the traditional ELM training is a combination of old and new training data,which takes too long time.This paper proposes some optimization methods based on different directions for OS-ELM algorithm.The research work done is as follows:First of all,this paper introduces kernel concept to OS-ELM,and makes the modified gauss and polynomial combined kernel function as the OS-ELM's kernel function.Then from the perspective of parameters optimization,the FOA is used for optimizing the parameters of the hybrid kernel function,and the FOS-ELM algorithm is proposed.Secondly,in order to solve the problem of ELM's output instability,the independent OS-ELM network is selected randomly and training with the same hidden nodes and output function,then we make majority voting class label as output.In order to improve learning efficiency of voting OS-ELM algorithm by introducing weights to distinguish the importance of each OS-ELM in decision making.At the same time an adaptive selective ensemble framework is designed to balance the accuracy and speed of the algorithm.And a novel FASEN algorithm is adapted to optimize these weights.Finally,the experiment results in UCI data sets demonstrate that the OS-ELM based on FOA has better approximation performance and good generalization performance.Meanwhile,we select the FOA algorithm to optimize the kernel parameters which could improve the convergence rate of original differential evolution algorithm,accelerated the speed of OS-ELM search parameters,and improved the classification accuracy.At the same time,the FVOS-ELM algorithm increases classification stability and classification accuracy which could enrich the theoretical research of online learning and provides a better reference for future practical application.