Recognition Of Handwritten Digits Based On Kernel Methods

Kernel based learning algorithms have gained more and more popularity in the machine learning community for its solid foundation, promising performance in recent years. In this article, kernel functions are formally discussed in mathematical terms and two of the kernel based learning algorithms: Support Vector Machine (SVM) and Kernel Fisher Discriminant Analysis (KFD) are introduced and applied to handwritten digits recognition.Based on analysis and comparison of the existing SVM training algorithms, especially SMO, a revised decomposition algorithm named GD is proposed. It balances well between the scale of the sub quadratic programming problem and the efficiency and times of iteration. Experiment shows that it can substantially reduce the training time of SVM with nonlinear kernels.Classic SVM can only deal with two-class problems, in this article several schemes for multi-calss problems are proposed and applied to handwritten digits recognition.Prior knowledge about the problem domain is an important way to improve the performance of the classifier. In this article, an implementation of incorporating prior knowledge into SVM is provided, which is with respect to the invariant transformations of translation and rotation of little angle in images of digits.Based on kernel trick, the classic Fisher's linear discriminant analysis and statistical uncorrelated discriminant analysis are generalized to kernel space. Experiment on handwritten digits shows that it performs as well as SVM.