Feature Extraction Methods Based On Improved PCA And LDA

Principal component analysis(PCA) and linear discriminant analysis(LDA) together with their related improvement methods are the commonly used dimension reduction approaches in the fields of machine learning and pattern recognition. They can effectively reduce the dimension of a given data set and avoid the problem of curse of dimensionality caused by the high-dimensional data set. However, the two methods both have shortcomings in practice. For example, the modeling speed of L1 norm based kernel principal component analysis(KPCA-L1) will be slow when it is used to deal with large scale data sets. Moreover, the traditional LDA is sensitive to noise because of using the L2 norm based distance measure. To enhance the modeling speed of KPCA-L1 and the anti-noise ability of LDA, the related research on the two feature extraction methods, i.e., KPCA-L1 and LDA, is conducted.1. A novelty detection method based on sample selection and weighted KPCA-L1 is proposed. It firstly selects the representative sample subset from a given training set. Then, the samples in the obtained subset are assigned with weights and the weighted KPCA-L1 is constructed. In comparison with KPCA-L1, the proposed method can efficiently reduce the scale of training set, accelerate the modeling speed of the feature extraction model, improve the update method of the KPCA-L1 algorithm, and to a certain extent, increasing the speed of the novelty detection. Experimental results on the synthetic and benchmark data sets demonstrate that, compared to KPCA-L1, the proposed method can obtain faster processing speed on the premise of assuming the accuracy rate of novelty detection.2. Linear discriminant analysis based on Lp norm(LDA-Lp) is proposed. The proposed method can obtain a set of local optimal projection vectors by maximizing the Lp-norm-distance-measure based ration between the between-class scatter and the within-class scatter, utilizing the gradient ascent method and the greed algorithm. In comparison with LDA, the proposed method can tackle the Lp norm based distance measure with p is taken as arbitrary values. Moreover, it can enhance the generality ability of LDA. Experimental results on the synthetic and benchmark data sets demonstrate that the proposed method exhibits better robustness.