Extreme Learning Machine Based Representation Learning

In such an era that information technology develops rapidly,data is cumbersome and complicated.As a consequence,how to fully analyze data,extract effective potential information and key features to learn the representation of data becomes one of the emphases for research in the field of machine learning.Existing representational learning algorithms can be divided into two categories: matrix decomposition based and neural network based algorithms.In recent years,with the increase in diversity,complexity and size of data,matrix decomposition based methods have obviously restricted the improvement of the results in the representation learning problems.Most researchers turned to deep learning based on neural networks for further study.At present,most deep learning algorithms,such as deep belief networks,convolutional neural networks,and recurrent neural networks,use iterative methods for updating parameters to train the model.It usually leads to many parameters,complex model,and low training efficiency.Extreme Learning Machine(ELM)is characterized by few training parameters,fast learning speed,and generalization ability.In view of the high efficiency of ELM,some researchers devote to studying how to integrate ELMs into the deep learning framework for alleviating the computational complexity and time-consuming problem of deep learning.In view of the deficiency of existing deep learning algorithms based on ELM that do not fully consider the global structure information,we first propose a new extreme learning machine autoencoder in combination with graph embedding framework(GDELM-AE),which can exploit both local near-neighbor structure and global structure information in ELM space.In GDELM-AE,intrinsic graph and penalty graph under the graph embedding framework are constructed by local Fisher discriminant analysis(LFDA).Furthermore,a stacked graph embedded denoising extreme learning machine(SGD-ELM)is proposed for feature representation by stacking several GDELM-AEs.The comparative results with state-of-the-art algorithms in multiple standard datasets show that the proposed SGD-ELM can obtain higher accuracy as well as faster training speed compared with the existing algorithms.It indicates that the proposed GDELM-AE can perform more comprehensive and noise-robust feature representation.SGD-ELM can easily obtain high-level abstract features.Existing ELM based deep learning algorithms usually neglect the spatial relationship of raw data,and traditional deep neural networks have the problems of too many parameters,training difficultly.For these problems,we propose multi-grained cascade of ada Boost based ELM(gcAWELM).We use adaBoost based ELM as a basic module to construct cascade structure for feature learning.Different ensemble ELMs trend to extract diversity features.And multi-grained scanning is to exploit spatial structure of the original data.Representation learning is performed by multi-grained scanning and cascade structure.The algorithm has a simple structure and can select the number of grains adaptively so that it has the advantages of less parameters and simple training.The results on different size of image datasets show that gcAWELM can achieve well performance in different learning tasks(image classification and face recognition)even with the uniformed parameter settings.