Dimension Reduction Method With Unsupervised Extreme Learning Machine

Extreme learning machine(ELM)was initially proposed for training single-layer feed-forward networks.Compared with the traditional feed-forward neural network learning algorithm,it not only have fast learning speed,but also can find the global optimal solution.It have proven to be an efficient and effective learning paradigm for pattern and regression.Though various ELM variants were proposed in recent years,most of them focused on the supervised learning tasks while little effort was made to extend it into unsupervised learning paradigm.In many real world applications,labeled data is usually expensive to obtain but unlabeled data is relatively easy to collect.Therefore,it is of great significance to put ELM into learning tasks with only unlabeled data.In this paper,we study the dimensionality reduction model based on extreme learning machine.It performs the following tasks:1.Subspace technology thinks that the high-dimension data approximately inhabit multiple linear subspaces.The samples belonging to the same class exist in a linear subspace while the samples from different classes exist in different linear subspaces.Therefore the multi-linear subspace structure of high-dimension data contain the discriminant information.In order to preserve it during the process of dimension reduction,we proposes a dimensionality reduction method,named dimensionality reduction with extreme learning machine based on subspace structure preserving.This method uses subspace technology to study the subspace structure of high dimensional data,and then maintains the subspace structure on the ELM projection.2.Manifold regularization have been have been widely used in dimensionality reduction,the main merit of it is to ensure that samples are in close proximity in the original space remained as in the new space.However,the class differentiation of the distance measurement between samples decreases with increasing dimension in high dimension space.To solve this problem,we improve the US-ELM based on learning the similarity of the samples,and ensure that if two samples are similar then they are close in the new space.The experimental results show that our method significantly outperforms the compared dimensionality reduction methods.3.Manifold and subspace structures have been widely used in dimensionality reduction methods.However,most of these methods only consider a single structure and may lose some of the discriminant information.In this paper,we take both of them into account and propose a new approaches for dimensionality reduction,we exploit a hybrid structure by intergrading the two structures and then preserve the hybrid structure into reduced subspace.The experimental results show that the method is superior to the method considering only a single structure.