Theory And Application Research On Extreme Learning Machine

In recent years the extreme learning machine (ELM) is a newly developed learning machine, which has been successfully in the function approximation, time series prediction and pattern recognition. Taking the single hidden layer feedforward neural network as an example, ELM only requires the activation functions of the network to be differentiable. The connected weights and bias of the network are random assigned.The output weights of the network can be obtained by using least-squares solution to minimize the error,so the parameters need not iteratively adjusting.Therefore,ELM achieves rapid learning.This algorithm has the characteristic of simple principle,quick learning and good adaptability.Although ELM has the advantage of principle and rapid learning,it does not take advantage of the discriminant information of data.At the same time, the network of ELM usually has a large hidde layer is very limited,it needs a the structure optimization method;and the compressed sampleing is a usefull way to make the data sparse,so combining the two methods is a novel research direction.Finally,there are so many requirements for online learning,we should expend the learning model to the online model.To solve these problems above, this paper takes a intensively research on the ELM, the main contributions is as follows:(1) ELM based on discriminant information is proposed. Two methods are used to learn discriminant information. The first scheme constructs a regularization term, which embedd discriminant information and structural information of data as a regularization term; The second one uses the linear discriminant analysis method to analyze the hidden layer of ELM for feature extraction and dimensionality reduction, so as to make use of the discrimination information of hidden layer.Both methods are taking well use of the discrimination information in data.(2) The ELM based on compressed sampling is proposed. The advantage of compressed sampling is that data can be obtained from very few measurement. Based on the advantage of high-dimensional, input structure and the hidden layer of network can be optimized. Through the two steps above, network structure of ELM can be good optimized, with the computational complexity of the algorithm reduced.ELM can achieve feature selection of data by compressed sampling.(3) The online ELM using ridgelet kernels is similarly propozed.There are two ways of online ELM. First of all is to construct a ELM network by ridgelet kernel function. The first way uses kernel online least squares method for online learning.The use of kernel function well improves online learning; The second way is updating the hidden layer real-time by using sliding window technology, based on the onlin linear discrimination analysis. The online learning is achieved by adding the discriminant information of data constantly.The research is supported by NSFC(61072108,60601029,60971112,61173090), new century excellent talents item(NCET-10-0668), Higher school subject innovation engineering plan (111plan), No. B0704and central university basic scientific research business expenses.