Research On Some Key Issues Of Kernel Methods And Their Applications In Face Image Analysis

As a nonlinear approach, kernel methods possess a sound foundation and anextensive application potential for nonlinear pattern classification tasks. They are char-acterized by two merits. First, they build a bridge between linearity and nonlinearity;next, they introduce a kernel function to avoid the curse of dimensionality withoutincreasing computational complexity. At present, sample reduction for support vectormachines (SVMs), kernel construction and multiple kernel learning are all key researchtopics in the field of kernel methods. In terms of SVM sample reduction, some re-searches aim at developing sample reduction approaches based on kernel clustering.In terms of kernel construction, few kernels are successfully constructed for given datafrom specific application backgrounds. In terms of multiple kernel learning, it wasdeveloped under the framework of SVMs, and so far, there have hardly been reportson multiple kernel learning for subspace analysis methods.The contributions of this dissertation include the following three aspects.Atop-downhierarchicalkernelclusteringapproachnamedkernelbisectingk-means(KBK) clustering algorithm is proposed, which tends to quickly produce balancedclusters of similar sizes in the kernel feature space. On this basis, we present theKBK-SR algorithm for SVM sample reduction, which integrates a modified versionof KBK with a subsequent sample removal procedure as a sampling preprocess-ing part for SVM training to improve the scalability. Theoretical analysis and experimental results both show that, with very short sampling time, our algorithmdramatically accelerates SVM training while maintaining high test accuracy.A systematic method to construct a new kernel for the kernel-based LDA meth-ods is proposed, which is good for handling illumination problem. The proposedmethod first learns a kernel matrix by maximizing the di?erence between inter-class and intra-class similarities under the Lambertian model, and then generalizesthe kernel matrix to our proposed ILLUM kernel using the scattered data interpo-lation technique. Experiments on face images under varying illumination showthat, the kernel-based LDA methods with our ILLUM kernel deal with the illumi-nation problem well in face image recognition. And in this sense, ILLUM kerneloutperforms popular kernels such as the linear kernel and the Gaussian RBF kernel.Multiple kernel discriminant analysis (MKDA) is proposed. We first present amultiple kernel construction method for kernel-based LDA, then through opti-mizing the maximum margin criterion (MMC) using the Lagrangian multipliermethod, deduce the weight optimization scheme for MKDA. In the experiments,on one hand, MKDA with optimized weights shows superior discriminant powerto KDDA with individual kernels on several UCI benchmark datasets; on the otherhand, the weight optimization scheme for MKDA is used to e?ectively select thekernel with the most discriminant power for recognition of face images.