Joint Sparsity And Local Linearity Based Extreme Learning Machine And Its Application

With the development of computer science and technology, artificial intelligence has become more and more important, which includes computer vision, data mining, robot, image classification, natural language processing, and so on. Among these fields, machine learning plays a key role, for it can provide algorithm support for other related fields. Artificial Neural Network(ANN) is one of most important machine learning algorithms, which can simulate the operation mode of biological neural networks and can implement classification and regression. In this paper, one of the ANN algorithms which is named as Extreme Learning Machine(ELM) is studied.ELM is one of the important machine learning algorithms, which is a neural network based on the hidden nodes. Compared to the traditional neural networks, ELM has the following advantages: 1) The training costs is very low; 2) It can obtain the global optimal solution; 3) The design is flexible and simple, which is easy for amateurs in the field of machine learning; 4) Various problems in machine learning such as compression, feature extraction, clustering, regression and classification can be solved in a unified framework. Therefore, this paper mainly focuses on both the theory and application of ELM. The main contribution is shown as follows:1) A joint sparse regularization based Sparse Semi-Supervised Extreme Learning Machine(S3ELM) algorithm is proposed. To ensure the classification accuracy as a precondition, the objective of S3 ELM is to stream the network size and speed up the algorithm. Different from previous pruning algorithms, S3 ELM is a non-greedy pruning algorithm developed based on the regularization, Moreover, the adding of semi-supervised regularization makes the algorithm perform better under the condition of fewer tag information. The experimental study on artificial data sets, UCI data sets and face data sets demonstrate that the proposed algorithm can obtain more simple structure and also can reduce the test time.2) The study was based on the developed theory that a linear separable data sets under an adequate linear tranformation can maintain the linear separable nature, In this study, a Local-Linear-Separability-based Extreme Learning Machine algorithm is proposed. Different from the traditional ELM in which the random hidden nodes are independent from the data, the new method relies on the distribution of data and keeps the feature of hidden nodes. As the random hidden nodes are related with data distribution, the proposed algorithm can achieve satisfying performance with less hidden nodes, which also reduces the complexity of structure of the network. The experimental results on six UCI data sets demonstrate that the two proposed algorithms can improve the classification performance, while reducing the network size and speed up the algorithms.3) A Clustering by Local Label Approximation with Extreme Learning Machine algorithm(LLAELM) is proposed. In this method, the kernel ridge regression classifier in local learning clustering is replaced by the ELM. Due to the advantages of ELM, the proposed algorithm can obtain better clustering result. The experimental results demonstrate that better clustering performance can be obtained after dimensionality reduction.(4) A Spatial-Neighborhood-based Semi-supervised Extreme Learning Machine method is proposed, which is applied to hyperspectral image classification. Due to the fact that the spatially adjacent pixels of hyperspectral image always belong to the same category, a corresponding mathematical model which is based on the graph Laplace regularization is developed. Moreover, an alternating direction iterative algorithm is developed. This method can directly transfer the information with labeled data to unlabeled data. The experimental results demonstrate that ELM can obtain the satisfying performance in both applications.