The Research On Support Vector Machine Ensemble Learning Approach

Support vector machine (SVM) is one type of learning machines that is paid wide attention in recent years. Based on statistical learning theory (SLT), SVM possesses many merits such as concise mathematical form, standard fast training algorithm and excellent generalization performance, so it has been widely applied in data mining problems such as pattern recognition, function estimation and time series prediction, et al. However, some problems, for example, the model selection, efficiency of SVM for large-scale training set, etc, still need to be solved in SVM research. Generally, almost all researches use single SVM as learner, and multi-SVM learner methods are scarce thought out. Ensemble learning technology as an effective multi-learner method has been obtained many valuable achievements. If the ensemble learning technology can be introduced to SVM, the generalization performance of SVM may be improved efficiently. Therefore, research on ensemble SVM learning becomes an important research issue. In the thesis, ensemble SVM learning method is investigated systematically. The main achievements are concluded in the following:(1) Research the theoretical analysis, the design of implementation method and the practical application of ensemble learning.(2) Introduce the existing methods of ensemble learning and analyze two classic ensemble learning methods: Bagging and Boosting; Compare the advantages and disadvantages of them and inspect their internal mechanisms.(3) Propose two SVM approaches, i.e. the regression SVM ensemble learning approach based on Bagging and the regression SVM ensemble learning approach based on parameter.(4) Present a regression SVM selective ensemble approach. By introducing three thresholds, some component SVM can be chosen, and the efficiency of the whole SVM ensemble learning system can be further improved. We analyze the impact of threshold changes on selective ensemble learning. Inspect the relationship between ensemble learning scale and selective ensemble learning scale, and analyze the relationship between ensemble learning scale and the effectiveness of selective ensemble learning.(5) Verify the three methods presented in the thesis on standard datasets and real-world datasets, and achieve expected experiment results.This thesis explores the ensemble SVM learning and selective ensemble SVM learning initially. The problem proposed in the thesis is a new issue for SVM research. Therefore, the obtained results not only have important theoretical significance, but also possess direct application value for real-world problems.