Research On Approximate Order Estimation Of Incremental Extreme Learning Machine

In recent years, Extreme Learning Machine (ELM) with fast learning speed and good generalization performance has appeared as a linear learning system through randomly selecting the hidden variables of a nonlinear learning system. Incremental Extreme Learning Machine (I-ELM) is a improved algorithm by gradually adding the hidden nodes to the networks, In the case of getting the same error, this algorithm has many advantages compared with other similar algorithms, such as timesaving, less hidden nodes and good generalization per-formance. However, it is familiar to us that the current theories merely from a qualitative point prove that this algorithm could approximate arbitrary continu-ous function with small error, which cannot explain the rapid essence of I-ELM. Therefore, it is quite meaningful to explore the approximation order of I-ELM, based on the theory of greedy algorithm, this thesis deeply focuses on the study of the approximation order of I-ELM, and presents the quantitative convergence analysis of this algorithm. The major works of this thesis includes:Chapter 1 first systematically introduces the origin and development of artificial neural networks (ANNs), then reviews the architecture, mathematical formulation and learning schemes of single-hidden layer feedforward networks (SLFNs), and finally describes some background knowledge of ELM.Chapter 2 mainly explains the design principles, the algorithm procedure as well as the merits and demerits of ELM with its significant development I-ELM.Chapter 3 focuses on some basic approximation theory about the continuous function, and introduces approximation thought and specific iterative process of the Greedy algorithm.Chapter 4 incorporates the idea of estimating the approximation order of Greedy algorithm into I-ELM algorithm. First construct a activation function sets, whose function satisfies specific conditions. Based on this, the concrete ap-proximation order of I-ELM to arbitrary continuous target function is proposed, and prove the major result in the form of theorem.