Research On Classification-oriented Subspace Analysis Methods And Their Applications

Feature extraction is the elemental problem in the area of pattern recognition. How to extract discriminant features is the key issue of pattern recognition. The subspace analysis methods are widely used for feature extraction due to their favorable properties, such as low computational cost and effectiveness. The essence of the subspace analysis methods is to reduce a high dimensional original space to a low dimensional feature subspace which is benefit for classification. At present, most of the subspace analysis methods aim to find the most discriminant features according to the patterns rather than the classifications. Therefore, the performances of the classifications will degrade when these features are used for classification. In this dissertation, starting from the classifications, we design three classification-oriented subspace analysis methods which extract the most discriminant features to the classifications. The proposed methods achieve remarkable results in the tasks of biometrics and handwritten numeral recognition.The main work and contributions of this dissertation are summarized as follows:(1) Based on linear regression classification (LRC), considering the reconstruction strategy and the classification rule of LRC, we analyze what is the optimal subspace for LRC Then we propose reconstructive discriminant analysis (RDA) to find the optimal subspace for LRC.In this paper, we prove that RDA also suffers from the small sample size (SSS) problem theoretically. Then we provide a way to solve this problem. Finally, we discuss RDA usually can extract N features, where N is the total number of the training samples. Since RDA are designed according to the reconstruction strategy and the classification rule of LRC, LRC works more effectively in the subspace of RDA. The experimental results demonstrate that the combination of RDA plus LRC outperforms other combinations of different feature extraction methods plus different classifications.(2) According to the classification rule of sparse representation-based classification (SRC) and LRC, we develop the maximum nearest subspace margin criterion (MNSMC). We also develop ridge regression classification (RRC) by using ridge regression instead of linear regression. Compared LRC, RRC can be applied in the case when the dimensionality of the original space is less than the training number of each class.Variations of illuminations and noises in the original space usually arouse larger intra-class reconstruction error and smaller inter-class reconstruction error, which leads to misclassifications when SRC or LRC is applied for classification. From the view of multi-subspace. MNSMC addresses this issue effectively by maximizing the heterogeneous nearest subspace margin and minimizing the homogeneous nearest subspace margin simultaneously. Compared with RDA, MNSMC avoids the SSS problem and can extract more features. The experimental results demonstrate that the combination of MNSMC plus LRC or SRC outperforms other combinations of different methods plus different classifications.(3) Based on discriminative locality alignment (DLA) and the classification rule of LRC, we proposed locally reconstructive patch alignment (LRPA). DLA is based on the idea of part optimization and whole alignment. However, the performance of DLA depends on the values of the model parameters heavily. Considering the classification rule of LRC, LRPA finds the subspace which minimizes the intra-class reconstruction error of each patch and maximizes the inter-class reconstruction error of each patch. The experimental results demonstrate that LRPA plus LRC is robust to the variations of the model parameters and achieves higher performance than DLA.