Research On KPCA Based On Block And Kernel Parameter Selection

Kernel methods have been widely used in pattern recognition and machine learning fields since 1990s. Kernel methods are non-linear methods based on non-linear mapping. Its implementation is equivalent to the implementation of a linear method in a high-dimensional feature space induced by a non-linear mapping. The outstanding advantage of kernel methods is that it provides the approach to apply linear analysis methods in feature space without the computation of mapping. Hence, feature extraction using kernel methods is more efficient than other general non-linear methods. On the other hand, kernel methods can effectively avoid the problem of"curse of dimensionality"compared to other non-linear methods. Kernel principal component analysis (KPCA) is one of the most frequent kernel methods used in pattern classification. It seems that two procedures are implicitly contained in the implementation of KPCA. The first procedure maps the original samples into the feature space and the second procedure carries out principal component analysis (PCA) in the feature space. Extracting features of samples which are not linearly separable, using KPCA can obtain better feature extraction performance than using PCA.Like PCA, KPCA is a holistic method. When employing KPCA to extract features, we can only extract holistic features but ignore the local features. Under the circumstance that the local information is of vital importance, using KPCA as the feature extraction method will discard lots of important local information. In this work, for the first time, we introduced block partition scheme into the conventional KPCA. This approach first divides each pattern into several blocks, then extracts features from each block. Block based KPCA (BKPCA) can effectively catch the local information through feature extraction on each block. To evaluate the effectiveness of BKPCA, we applied this approach to face recognition under variable environment. Experiments on three well-known face databases illustrate that BKPCA can effectively reduce the effect of variable environment on face recognition. Hence, BKPCA is an effective approach when extracting local features from patterns.Given a kernel function, there is at least one kernel parameter. KPCA can be implemented only if a value is assigned to the parameter. Because different parameter values usually produce different feature extraction results, choosing an appropriate parameter deserves further study. In this work, we explored the relationship between the feature extraction performance of KPCA and the ratio of the eigen-value of the eigen-equation on all samples to the summation of the eigen-values of the eigen-equations on all subclasses. We refer to this ratio as eigen-ratio. Our experiments show that the classification result is closely related with eigen-ratio. Hence, we can tune the parameter value of KPCA used for classification through maximizing the eigen-ratio.