Research On Enhanced Canonical Correlation Analysis With Applications

Machine learning models the problem at hand using the finite observational data with the help of the data analysis tools to reveal the underlying relationship among the observations. Canonical correlation analysis (CCA) acts as a powerful tool to analyze the underlying dependency between the observed samples in two sets of data. Initially proposed in 1936 as the multivariate analysis method, CCA has been widely employed in regression and modeling, image analysis and processing, computer vision, pattern recognition, bioinformatics and etc. As a result, CCA has gained more and more attention of the researchers in the related research fields; in addition, the emerging multimodal recognition techniques offer new opportunities for the CCA based recognition algorithms.In this dissertation, we focus on the two principal problems in machine learning, i.e., regression and pattern recognition, using the proposed enhanced CCA models. The main contributions of this dissertation are summarized as follows:(1) Local preserving CCA (LPCCA) is proposed as a nonlinear extension of CCA to adapt to the nonlinear correlation in real applications. The globally nonlinear problem is decomposed into a serial of locally linear sub-problems. The proposed method is validated through the experiments of both the data visualization and the pose estimation.(2) A simple unified framework is constructed for the unimodal recognition using CCA to reveal the underlying mechanism of equivalence between the class-label-based CCA and linear discriminant analysis (LDA). Moreover, we propose CCA based on the independent soft label for each sample rather than a class to break the limitation of recognition performance due to this equivalence.(3) A novel supervised learning method, termed as discriminative CCA (DCCA), is proposed, which embodies the impacts of both within-class correlation and between-class correlation on classification. With the help of kernel trick, the kernelized discriminative CCA (KDCCA) is further proposed to tackle the linearly inseparable cases. The experiments show the superiority of both DCCA and KDCCA to other relatived methods in terms of the recognition performance. (4) Based on DCCA, the discriminative CCA with missing samples (DCCAM) is further proposed to overcome the difficulties due to the loss of samples in real applications. Besides the inherited advantages from DCCA, DCCAM possess the characteristics of better recognition performace,timesaving, space-saving and relatively insensitive w.r.t. the number of the missing samples.(5) The idea of correlation as the similarity metric between the samples in the context of CCA is generalized to principal component analysis (PCA), and the correlation based pseudo-principal component analysis (p-PCA) is proposed, Moreover, this idea is generalize to the recently developed algorithms of matrix-pattern-based PCA family and better recognition performance can be achieved. On the other hand, another supervised learning method, the class-information-incorporated PCA, is proposed without change of the original PCA framework, and the experiments validate the proposed method.