The Efficient Kernel Method Based On Sparse Model

Kernel methods have been widely used in pattern recognition and machine learning fields since 1990. The outstanding advantage of kernel methods is that it provides the approach to apply linear analysis methods in feature space without the computation of mapping. The other advantage of kernel methods is that it avoids the"curse of dimensionality". However, it should be figure out that we need compute the kernel functions regarding all the training samples when kernel methods are employed as the feature extraction methods. Hence, the size of training set constrains the feature extraction efficiency of kernel methods. In some real time computation fields,the kernel methods even can not be applied. It is significant to improve the feature extraction efficiency of kernel methods.According to the idea of significant nodes that are selected from training samples, this paper develops several kinds of sparse kernel methods. In the sparse kernel methods, we need compute only the kernel functions that regarding to the testing sample and the significant nodes. As a result, the most contribution of the derived method is its feature extraction procedure that is much more computationally efficient than the naive kernel method. For the significant node selection methods, this paper gives the network model responding to KMSE. According to the weight of the network, we design an algorithm to select the significant nodes that contribute most to the feature extraction results in training samples, and then we design the sparse KMSE model. We also reveal that the discriminant vector of sparse kernel can be represented by the linear combination of significant nodes in feature space, and this deduce can be used to guide the construction of the other sparse kernel models.Although the KMSE is a nonlinear analysis method, it can be viewed as the linear regression in feature space. Regression analysis methods provide the criterion to estimate the contribution of each variable, and this paper applies the linear regression analysis methods to construct another sparse KMSE.Besides the KMSE, the other kernel methods, such as KFDA, KPCA, are facing the same problem in feature extraction efficiency. We also find an algorithm that is appropriate for the most of kernel methods, which is independent to the decision algorithm. According the deduce that the discriminant vector of sparse kernel can be represented by the linear combination of significant samples, this paper designs the sparse KFDA and sparse KPCA, respectively. Experimental results on several benchmark datasets illustrate our methods can efficiently classify the benchmark data with the high recognition accuracy.