Researches On Support Vector Machine Learning Approaches Based On Ensemble Learning

Support vector machine(SVM),an effective tool in the area of machine learning, has aroused considerable attention with its convenient training approach and good generalization performance and has achieved gratifying results in many fields such as function estimation, pattern recognition. However, as the development of information technology, machine learning is facing an increasing data dimension and scale, so performance and efficiency of SVM are affected in a certain extent.In addition, the performance of SVM is influenced easily by kernel functions and parameters, so it is still a hot issue in SVM researches to find an efficient and stable method for kernel selection.Ensemble learning, with favorable stability and good generalization ability, is a typical multi-machine learning approach. In order to solve the above problems,SVM and ensemble learning techniques are integrates effectively in this thesis, and the researches are concluded in the following:(1)Introduces the basic principles and classical algorithms of ensemble learning systematically, and analyzes the characteristics of ensemble learning and the advantages of bringing ensemble learning to SVM researches.(2) Proposes a feature selection algorithm on SVM based on ensemble learning by combining Bagging with multiple of feature selection algorithms. At first, a number of training subsets are produced by bagging method, and then the different feature subsets obtained from corresponding feature selection algorithms respectively are used to training individual SVM leaner as input space in each training subset. Different feature selection algorithms can make the diversity among ensemble individuals larger and improve system performance, so the shortage that best feature subset is difficult to obtain can be avoided. Experimental results on UCI data sets indicate that, compare with ensemble learning based on single feature selection algorithm, the approach this thesis proposed could make the diversity among the individuals lager and improve the learning performance more.(3)Proposes a large-scale data processing algorithm on SVM based on ensemble learning by combining Bagging with clustering algorithm, the basic idea of the algorithm is producing the training subsets by extracting small-scale samples from large-scale dataset to training the individual SVM learner. In order to make sure the extracted samples contain more information, the original dataset should be clustered at first, and then the samples are selected form each category in proportion. The experimental results in real dataset about air quality forecast and standard datasets show the effectiveness of the algorithm.(4) Proposes a kernel selection algorithm based on SVM ensemble.The algorithm constructs the SVM individuals with different kernel functions and parameters and integrates kernel selection with ensemble leaning, so it can be avoided that traditional SVM must select the kernel function and parameters first in dealing with practical problems.In this thesis, ensemble learning is used to SVM researches about feature selection, large-scale data processing, kernel selection.The research results are complements of SVM learning approaches, and provide a new model for solving practical application by SVM.