Font Size: a A A

Research On Feature Transformation Based On Kernel Principal Component Analysis

Posted on:2015-08-29Degree:DoctorType:Dissertation
Country:ChinaCandidate:S K YangFull Text:PDF
GTID:1228330467989096Subject:Circuits and Systems
Abstract/Summary:PDF Full Text Request
As a typical case of the popularity of kernel methods in the field of machine learn-ing, feature transformation based on kernel principal component analysis (KPCA) has been widely used in many applications. Due to its outstanding performance, there has been growing interest about KPCA, which has become one of the research focus in ma-chine learning and pattern recognition. With the development of information technology, machine learning has become an accelerator powering the information technology revolu-tion, which will promising a broader range of applications of feature transformation based on KPCA. However, there are still many issues to be resolved in both theory and appli-cations of feature transformation based on KPCA, such as the robustness of the feature transformation, the effectiveness of feature extraction, solution to the pre-image problem, etc. Therefore, research on this topic is of great significance.In regard to the existing main problems in feature transformation based on KPCA, this paper conducted a series of studies, proposed a number of feature transformation and its optimization methods based on KPCA, including:1. Feature transformation based on single subspace KPCAUsing heartbeat classification as the application background, the feature transfor-mation based on single subspace KPCA is studied in this paper. Along with discrete wavelet transform and neural network, a complete automatic heart beat classifica-tion system is presented. This system consists of three parts:signal preprocessing, feature transformation and pattern recognition. As the critical part, feature trans-formation based on KPCA gives a more complete nonlinear representation of the heartbeats. The experiments show that the proposed heartbeat classification system achieves very good classification results under both the class-oriented and subject- oriented evaluation, suggesting a better performance than the state-of-art method in real situation.2. Feature transformation based on multi-subspace KPCAFor the disadvantage of the feature transformation based on single subspace KPCA that the inter-class characteristic differences can not be effectively retained, using face recognition as the application background, a feature transformation method based on multi-subspace KPCA is proposed in this paper, by introducing the cat-egory information in the calculation of kernel eigenfaces, and accordingly a face recognition method is proposed. Two versions of algorithm are realized based on KPCA and its variant-kernel entropy component analysis, and the minimum re-construction error is used as the classification rule. Experimental results show the clear advantage compared with the feature transformation based on single subspace methods. In addition, as for the choice of kernel functions, the polynomial kernel function is better than the radial basis function with respect to classification accu-racy.3. Feature transformation from feature space to input spaceFeature transformation from feature space to input space, which is also called "pre-image problem", is an important issue involved in kernel methods. However, it is an ill-posed problem, as the solution is usually nonexistent or not unique. In this pa-per, a novel method for solving the pre-image problem is proposed. In the proposed algorithm, an inverse mapping process is constructed based on a novel framework that preserves local linearity. In this framework, a local nonlinear transformation is implicitly conducted by neighborhood subspace scaling transformation to preserve the local linearity between feature space and input space. By extending the inverse mapping process to test samples, we can obtain pre-images in input space. The proposed method is non-iterative, and can be used for any kernel functions. Ex-perimental results based on image denoising using KPCA show that the proposed method outperforms the state-of-the-art methods for solving the pre-image problem. 4. Robustness feature transformation based on KPCAKPCA is quite sensitive to outliers, which makes the feature transformation lack of robustness. In this paper a robust KPCA algorithm is proposed. We follow the observation in the classical KPCA that the eigenvectors and central point can be ex-pressed as linear combination of training samples. Borrowing the idea from robust statistics, a robust loss function is induced. Finally the solution can be obtained in an iterative way. The proposed algorithm can make robust estimation of both the cen-tral point and the eigenvectors. The form of the solution to the proposed algorithm is of the same as that to the classical KPCA algorithm, thus it is easy to integrate into existing systems. Experimental results show that the proposed algorithm can quickly converge, and is robust to outliers.
Keywords/Search Tags:Kernel Principal Component Analysis, Feature Transformation, Kernel Eigen-face, Pre-image Problem, Robust Kernel Principal Component Analysis, Heartbeat Clas-sification, Face Recognition, Image Denoising
PDF Full Text Request
Related items