Feature Extraction Based On Subspace Decomposing And Kernel Space Mapping

The real world is infused with all sorts of redundant information with the fastexploration of Internet. How to extract meaningful stuff from the sea ofinformation is becoming increasingly essential to all of network users. Featureextraction aims to remove useless data among all the knowledge we have andretain those truly important. Traditional methods mainly focus on decreasing thedimensionality of feature vector. While this paper makes contributions in thefollowing aspects: Based on the traditional subspace decomposition method, anew method was introduced to solve the noise problem. Two subspaces wereconstructed to replace the original high dimensional space, thus successfullypreserved the largest variances and made a reasonable estimation about thecomplementary subspace. In this way, the impact of random noise was reduced tominimum.Principle Component Analysis (PCA) is known as K-L Transform or HotellingTransform. It is a standard technique commonly used for data reduction instatistical pattern recognition and signal processing. It is also a transform bywhich the data set can be represented by reduced number of effective features andstill retain the most intrinsic information content. A small set of features to befound to represent the data samples accurately. Also called "SubspaceDecomposition". – PCA is an optimal transform for signal representation anddimensional reduction, but not necessary for classification tasks, such as speechrecognition – PCA needs no prior information (e.g. class distributions) of thesample patterns.Linear discriminant analysis (LDA) is a traditional statistical method which hasproven successful on classification problems. The procedure is based on an IV吉林大学硕士学位论文eigenvalue resolution and gives an exact solution of the maximum of the inertia.But this method fails for a nonlinear problem. In this paper, we generalize LDA tononlinear problems and develop a Kernel Optimal Iterative Discriminant Analysis(KOIDA) by mapping the input space into a high dimensional feature space withlinear properties. In the new space, one can solve the problem in a classical waysuch as the LDA method. The main idea is to map the input space into aconvenient feature space in which variables are nonlinearly related to the inputspace. This fact has been used in some algorithms such as unsupervised learningalgorithms and in support vector machine (SVM). In our approach, the mapping isclose to the mapping used for support vector method, which is a universal tool tosolve pattern recognition problems. In the feature space, the SVM method selectsa subset of the training data and defines a decision function that is a linearexpansion on a basis whose elements are nonlinear functions parameterized by thesupport vectors. SVM was extended to different domains such as regression andestimation. The basic ideas behind SVM have been explored by Sch?lkopf et al.to extend principal component analysis (PCA) to nonlinear kernel PCA forextracting structure from high dimensional data set. They mention that it would bedesirable to develop nonlinear form of discriminant analysis based on kernelmethod.. The foundations for the kernel developments described here can beconnected to kernel PCA. Drawn from these works, we show how to express theKOIDA method as a linear algebraic formula in the transformed space usingkernel operators. Instead of trying to decrease the dimensionality of featurevectors, we utilized a powerful trick named kernel space method, which hasproved its effectiveness when used in Support Vector Machine. By constructing avery high dimensional (probably infinite dimensionality) space, and performingoptimal iterative linear discriminant analysis in this space, we can convert thenonlinear problems in original space to linear ones in kernel space.LDA is a standard tool for classification. It is based on a transformation of theinput space into a new one. The da...