Linear Projection Analysis: Theory, Algorithms, And Application In Feature Extraction

Feature extraction is the elementary problem in the area of pattern recognition. It is the key to solve the problems such as face identification and handwritten character recognition. Linear projection analysis, including principal component analysis (or K-L transform) and Fisher linear discriminant analysis, is the classical and popular technique for feature extraction. In this paper, we focus our attention on linear projection analysis and develop some new algorithms as regards it. And, these algorithms are verified to be effective in the application of face identification and handwritten character recognition.We first develop a theoretical framework for the uncorrelated Fisher linear discriminant analysis (ULDA) and show it to be an improvement of the classical linear discriminant analysis in theory. We demonstrate that ULDA outperforms the Foley-Sommon discriminant analysis (FSLDA) and discuss why it is. The inherent relationship between Fisher linear discriminant analysis and Karhunen-Loeve expansion is revealed, i.e., ULDA is essentially equivalent to one classical K-L expansion method. Moreover, we enhance ULDA using the idea of another K-L expansion method, and finally an optimal K-L expansion method is developed. The proposed method is tested on Concordia University CENPARMI handwritten numeral database, and the experimental results indicate that it is more effective than the other K-L expansion methods.How to get the optimal Fisher discriminant vectors efficiently in singular case is a very difficult and critical problem. In this paper, we try to solve this problem in theory. Our main idea is to map the singular problem in high-dimensional space into a nonsingular problem in reduced dimensional space without any loss of optimal discriminatory information with respect to Fisher criterion. Based on this idea, a general framework for linear analysis in singular case is developed, i.e., PCA is first used to reduce the dimension of image space to m (the rank of the total scatter matrix). Then, LDA is followed and employed for the second feature extraction in the transformed space. Based on this framework, a combinatorial optimal LDA (OLDA) algorithm is proposed. Experiments is performed on ORL face image database as well as NUST603 face image database, and the experimental results indicate that OLDA is robust, efficient and superior to the previous LDA-based methods in terms of recognition accuracy.The conventional principal component analysis (PCA) and Fisher linear discriminant analysis (LDA) are based on vectors. That is to say, if we use them to deal with the image recognition problem, the first step is to transform original image matrices into same dimensional vectors, and then rely on these vectors to evaluate the covariance matrix and to determine the projector. The drawback of this strategy is obvious. Firstly, to perform PCA or LDA on basis of such high-dimensional image vectors is a time-consuming process. Secondly, the high dimensionality usually leads to singularity of the within-class covariance matrix, which is a trouble for calculation of Fisher optimal discriminant vectors. Rather, in this paper, two straightforward image projection techniques, termed image principal component analysis (1MPCA) and image Fisher linear discriminant analysis (IMLDA), are respectively developed to overcome the weakness of the conventional PCA and LDA as applied in image feature extraction. Their main idea is to directly construct image covariance matrices based on images themselves, and them utilize them to perform PCA or LDA. Since the size of image covariance matrices is same as that of images and the within-class image covariance matrix is usual nonsingular, thus, the difficulty resulting from high dimensionality and singular case are artfully avoided. Experimental results on ORL face database show the proposed IMPCA and IMLDA are more effective and efficient than conventional PCA and LDA based methods such as Eigenfaces and Fisherfaces.Finally, a strategy of feature parallel fusion is develope...