Theory Researches And Applications Of Multi-view Canonical Correlation Analysis

Multi-view data are common in real-world applications,and feature extraction researches based on multi-view data are becoming popular and important in pattern recognition.As a very important multi-view feature extraction method,multi-view canonical correlation analysis(MCCA)has received intensive attention at home and abroad.However,data collected from real-world applications are usually complicated and nonlinear,and MCCA is difficult to deal with such data,which will greatly limit the performance of MCCA and prevent the extension of MCCA.Therefore,by means of multi-patch embedding,graph enhancement approach,label kernel technique,fuzzy projection,kernel aligenment,super-resolution reconstruction,and so on,this paper explores multi-view canonical correlation alaysis theories and has proposed some effective multi-view feature extraction methods.These proposed methods can effectively solve some problems of real-world applications.Our work mainly includes the following parts:(1)Multi-patch embedding multi-view canonical correlation analysis(MEMCCA)is proposed.In existing locality-based correlation anlaysis methods,single-category local neighbor relationships are difficult to well reveal intrinsic manifolds of raw high-dimensional data,and holistic locality information will be lost when supervised information is incorporated into these methods.Therefore,on the basis of local patches,we explore multi-source patch structures and multi-source fusion theories under the multi-view correlation analysis framework,and further propose the MEMCCA method.The proposed method is able to automatically learn the holistic locality information,and the learned information is more beneficial for correlation feature extraction.Moreover,the proposed method further improves the class separability of correlation features with the help of within-view scatter structures.From extensive experiment results on NIR images,face images,and multi-view car images,it can be seen that MEMCCA possesses superior performance of image recognition and good parameter robustness.(2)Graph-enhanced multi-view discriminant correlation analysis(GMDCA)is proposed.Graphs in multi-view correlation analysis methods are difficult to well capture intrinsic geometry structures hidden in raw data.Therefore,we present a graph enhancement approach that includes space partitioning technique of data and a supervised probability integration model.Moreover,enhancement graphs constructed by this approach can better reveal intrinsic geometry structures and discriminative distribution information of raw data.On the basis of the enhancement graphs,we further explore effective incorporation technique of graphs and class labels under the multi-view correlation analysis framework,and then propose the GMDCA method.In the proposed method,we characterize intraclass and interclass enhancement correlations of between-view correlation features,and simultaneously consider a global intraclass scatter that can better capture within-view and between-view scatter structures.Extensive experiment results have demonstrated the effectiveness of the graph enhancement approach and the superior recognition performance of GMDCA.(3)A novel label kernel is developed and a multi-view label kernel correlation analysis(MLKCA)algorithm is proposed.Aiming at the lack of supervised information in empirical kernel methods,we propose a novel label kernel method(LKM)that can well preserve the discriminating power of class labels with the help of a label-based unit hypersphere model.Due to the dependence on class labels,LKM is hard to deal with out-of-sample data.Therefore,according to similarity principle of sample distribution,we give a projection strategy for assisting LKM,i.e.fuzzy projection strategy(FPS).Then,we naturally integrate LKM and FPS into multi-view correlaiton analysis theories,and further propose the MLKCA algorithm.The proposed algorithm is able to learn nonlinear correlation features of multi-view data,and the learned features have well discriminating power with the help of LKM and FPS.Additionally,on five different datasets,we analyze the influence of the neighbor parameter on fuzzy projection and the recognition performance of MLKCA.(4)Kernel-aligned multi-view canonical correlation analysis(KAMCCA)is proposed.It is difficult to obtain one suitable kernel for one view,and only one single kernel in one view has some limitations.Therefore,we transform each original feature vector into a 2-dimensional feature matrix by means of kernel alignment,and then propose the KAMCCA method.The proposed method can simultaneously employ numerous kernels to better characterize data distribution information of each view,and a data-dependent mixture kernel can be automatically learned for each view.Moreover,the kernel extension technique of KAMCCA can be used for extending many other feature extraction methods to their multi-kernel variants,and the multikernel extensions are easy by means of the technique.Based on NIR face images,thermal face images,visible face images,handwritten images,and object images,we design extensive experiments,and extensive experiment results have manifested the effectiveness of KAMCCA in image recognition performance.(5)Super-resolution based on local multi-view coherent subspace learning(LMCSL-SR)method is proposed.To better satisfy the similar locality assumption of super-resolution reconstruction,we give a local multi-view coherent subspace learning approach;to simultaneously employ multi-category low-resolution images to reconstruct high-resolution images,we present a multiple correlation fusion strategy;to determine more accurate neighborhoods,we design a reconstruction revision strategy.Then,based on these improvements,we develop a novel face super-resolution method,i.e.the LMCSL-SR method.The method solves some key problems that cannot be solved in existing super-resolution methods based on correlation analysis theories.Additionally,LMCSL-SR is the first method that realizes multi-source reconstruction of image super-resolution under the correlation analysis framework.For analyzing the necessity of residual compensation,the effectiveness of the multiple correlation fusion strategy,the superiority of the local multi-view coherent subspace,the influence of parameters,the influence of the training image number,and the quality of final reconstruction images,we design extensive experiments,and all the experimental results can give a reasonable observation that LMCSL-SR is an effective face super-resolution reconstruction method.