Image Dimensionality Reduction And Object Recognition Based On Sparse Learning

In many fields of scientific research such as face recognition, bioinformatics, andinformation retrieval, the data are usually presented in a very high dimensional form.This makes the researchers confront with the problem of “the curse of dimensionality”,which limits the application of many practical technologies due to the heavycomputational cost in high dimensional space, and deteriorates the performance ofmodel estimation when the number of training samples is small compared to the numberof features. How to find an effective low dimensional representation which meets theneeds of real-world applications has aroused considerable interests in the fields ofpattern recognition, machine learning, data mining and computer vision.In practice, dimensionality reduction has been employed as an effective way todeal with “the curse of dimensionality”. In the past years, a variety of dimensionalityreduction methods have been proposed. However, many popular dimensionalityreduction methods have some limits. PCA is a good method for representation, but itdoes not consider the label information. Thus it may not be reliable for classification.Because the label information is employed, LDA has proven to be more effective thanPCA in classification. However, LDA can extract at most K-1features (K is thenumber of classes), which is undesirable in many cases. Moreover, it confronts theSmall Sample Size (SSS) problem when dealing with high dimensional data. Popularlocal neighborhood structure preserving method LPP relies on artificially pre-definedneighborhood graph. The computational complexity of the recently proposed methodSPP is too high. Therefore, for large scale problem, SPP is computationally prohibitive.Based on the idea of sparse representation, this dissertation proposed several supervisedand semi-supervised dimensionality reduction methods in order to fast learn an effectivelow dimensional representation of high dimensional data and a robust object recognitionmethod, and applied the proposed methods to several real-world problems, such as facerecognition, document classification, remote sensing target recognition, etc. The maincontributions of this dissertation can be summarized as follows:1. We propose two novel efficient dimensionality reduction methods named FastSparsity Preserving Projections (FSPP) and Fast Fisher Sparsity Preserving Projections(FFSPP), respectively, which aim to preserve the sparse representation structure in highdimensional data. Unlike the existing Sparsity Preserving Projections (SPP), where thesparse representation structure is learned through resolving n (the number of samples)time-consuming1norm optimization problems, FSPP constructs a dictionary through class-wise PCA decompositions and learns the sparse representation structure under theconstructed dictionary through matrix-vector multiplications, which is much morecomputationally tractable. FFSPP takes into consideration both the sparse representationstructure and the discriminating efficiency by adding the Fisher constraint to the FSPPformulation to improve FSPP’s discriminating ability. Both of the proposed methods canboil down to a generalized eigenvalue problem. Experimental results on publiclyavailable face data sets and a standard document collection validate the feasibility andeffectiveness of the proposed methods.2. We propose a novel dimensionality reduction method called SparseRegularization Discriminant Analysis (SRDA), which aims to preserve the sparserepresentation structure of the data when learning an efficient discriminating subspace.More specifically, SRDA first constructs a concatenated dictionary through class-wisePCA decompositions which conduct PCA on data from each class separately, and learnsthe sparse representation structure under the constructed dictionary quickly throughmatrix-vector multiplications. Then SRDA takes into account both the sparserepresentation structure and the discriminating efficiency by using the learned sparserepresentation structure as a regularization term of linear discriminant analysis. Finally,the optimal embedding of the data is learned via solving a generalized eigenvalueproblem. The extensive and promising experimental results on publicly available facedata sets validate the feasibility and effectiveness of the proposed method.3. To tackle the problem of single labeled image per person face recognition, asubspace label propagation and regularized discriminant analysis based semi-superviseddimensionality reduction method (SLPRDA) is proposed. First, a label propagationmethod based on subspace assumption is designed to propagate the label informationfrom labeled data to unlabeled data. Then, based on the propagated labeled dataset,regularized discriminant analysis is used to conduct dimensionality reduction. Finally,the recognition of testing face is completed in low dimensional space using nearestneighbor classifier. Moreover, in order to better deal with the nonlinear structure in data,the nonlinear version of the proposed method is derived via kernel method. Theextensive experiments on three publicly available face databases validate the feasibilityand effectiveness of the proposed method.4. We propose a novel semi-supervised dimensionality reduction method, namedDouble Linear Regressions (DLR), to tackle the single labeled image per person facerecognition problem. DLR simultaneously seeks the best discriminating subspace and preserves the sparse representation structure. Specifically, a Subspace Assumption basedLabel Propagation (SALP) method, which is accomplished using Linear Regressions(LR), is first presented to propagate the label information to the unlabeled data. Then,based on the propagated labeled dataset, a sparse representation regularization term isconstructed via Linear Regressions (LR). Finally, DLR takes into account both thediscriminating efficiency and the sparse representation structure by using the learnedsparse representation regularization term as a regularization term of linear discriminantanalysis. The extensive and encouraging experimental results on three publicly availableface databases demonstrate the effectiveness of the proposed method.5. A new remote sensing target recognition method based on extending training setby rotation and sparse representation (RETSRC) is proposed for remote sensing imagerecognition which includes incomplete images. First, the training set is extended byrotating every image to guarantee the test image can be expressed approximately as asparse linear combination of the extended training set. Then, based on the sparserepresentation computed by1norm minimization, the test image is classifiedaccording to the degree of approximation to test image of the sparse representationcorresponding to different classes. The experimental results show that the proposedalgorithm surpasses the existing methods in recognition rate, is robust to incompleteimages and still shows good results under the condition of small scale samples.