Research For Support Vector Machines And Its Application In Pattern Recognition

SVMs (Support Vector Machines), which is based on Vladimir N.Vapnik's Statistical Learning Theory, is the most advanced machine learning algorithm in the field of pattern recognition. In this thesis, some topics in SVMs and its application in pattern recognition is researched, and we will focus on some key topics described as following:(1) Research for the realization algorithm of SVMs.(2) Research for the generalization ability of SVMs.Nowadays, just as the other algorithm to solve the pattern recognition problems, SVMs faces on the problem of parameter selection and optimization.How to select and optimize the kernel function, which determines the generalization ability of SVMs,is still an open problem,and this problem hampers the further development of SVMs. Recently, David McAllester et.al combined the Bayesian and PAC theory and presented some PAC-Bayesian theorems that bound the generalization error of Bayesian classifiers. Based on David McAllester's theory and geometrical arguments, Ralf Herbrich et al presented a margin bound for linear classifiers,and this bound can be used to optimized the hyperparameter of RBF kernel function.A tighter PAC-Bayesian bound for linear classifiers than which presented by Ralf Herbrich et al is presented in this paper, and the new bound can also be used to optimize the parameters of RBF kernel in SVMs. (3)Research for multi-class classifier based on SVMs.SVMs is a binary classfier, however, in practice, multi-class classifier is used. So, constructing multi-class classifier based on SVMs is a widely researched topic. Considering the requirement in practice, the multi-class classifiers based on SVMs should have good generalization ability, and it also should be modified according to data distribution and the time cost as well as the space cost should be acceptable during the course of classifier modifying. We use an algorithm, named ISUOA(Integrate SVMs with UCOA ),which integrates the idea of UCOA (UltraConservative online Algorithm) with SVMs, to construct multi-class classifier . During the course of modifying the classifier , only the Support Vectors (SVs)of relevant SVMs and the samples that are classified incorrectly are used as the training samples .The experiment result shows that, the multi-class classifier constructed using this algorithm has good performance.