The Study On Support Vector Machine Ensemble Learning

A team of researchers led by V.Vapnik at AT&Tbell laboratory, started the study of a small statistics from the sixties last century, and propounded Statistical Learning Theory (SLT). Support Vector Machine (SVM) is a new pattern recognition method developed in recent years based on SLT. SVM based on the principles of VC (Vapnik-Chervonenkis) dimension and structure risk minimization, and shows the advantages of capability in dealing with scared samples, nonlinear and high dimensions. SVM has successful application in text classification, image recognition and biological information processing. Since the nineties last century ensemble learning has become the focus of research in machine learning. The experimental measurement results indicate that the selective ensemble can achieve comparatively good generalization ability under the condition of a relatively small ensemble size. The main objective of selective ensemble is retaining based learners with large diversity to ensemble. At present, researches on the diversity focused two areas, the first is to find a suitable measure of diversity, and the second is how to enhance ensemble learning performance with this measure.This paper deals with the measures of diversity and selective ensemble based on SVM. Due to simple formation, diversity of SVM can be defined on the characteristics of their own. In this article we introduce a novel measure to quantify diversity for SVM, which is based on the characteristic parameters of support vector rather than validation set, and the clustering technology is used to enhance the diversity of individual. We studied the performance and relationship between diversity and accuracy by the method. In terms of these works above, we provide a selective classifier ensemble algorithm based on subgraph. The new algorithm is proved to achieve better or equivalent performance than the traditional ensemble classification algorithm. At last, in this paper we study Multi-class Support Vector Machines.