Research On Boost-SVM Algorithm Based On Ensemble Learning

Ensemble learning is a new machine learning paradigm. It can significantly improve the generalization ability of learning systems through utilizing multiple learners to solve a problem. Therefore the research on its theory and algorithm becomes a hot topic in the field of machine learning since the beginning of 1990s. However ensemble learning is till very young and a lots of problem of ensemble learning need be figured out. To solve practical classification problems, improving learning accuracy is a necessary way to decrease damage or mistakes in reality. But In these projects, such as planet exploration, seismic wave analysis, biology feature recognition, and computer aided medical diagnoses, and so on, ensemble learning can not meet their needs to complete missions in high quality. To make it done, we propose a new method—Boost-SVM, which puts SVM into AdaBoost framework to improve the learning accuracy of the SVM algorithm. The reason using Support Vector Machine (SVM) is that it base on statistical learning theory, and it solves model selection, overfitting, nonlinearity, dimensional disaster, local minimum value and so on.As we all know, classification mistakes is due to the samples are classified into the wrong class. Actually Boost-SVM method is aim to reduce the number of these wrong classified samples. By changing the weights of the training examples in the re-sampling process of Adaboost, there is less wrong classified examples in Boost-SVM. Boost-SVM algorithm is developed based on LibSVM platform. And it can implement to both alternative classification and multi-classification. Empirical results indicate that Boost-SVM effectively improves learning accuracy with competitive learning ability. Meanwhile Boost-SVM is much intelligible, simple than others.