Bayesian Model Averaging For Support Vector Machine On Regularization Path

The tuning of (hyper)parameters is an essential ingredient and important processfor learning and applying Support Vector Machines (SVM). Model selection is an indispensablestep to guarantee generalization of SVM. Combination of support vectormachines has a higher generalization than single SVM, but usually has low computationalefficiency. To solve this problem, We propose a novel probabilistic modelcombination method for support vector machine on regularization path (PMCRP). Ourmain work is:1. We develop an efficient regularization path algorithm, namely the regularizationpath of support vector machine based on positive-definite kernel (PDSVMP).2. We design the PMCRP based on the regularization path. The initial candidatemodel set is constructed by PDSVMP, and the combination on the initial modelsis achieved using Bayesian model averaging.3. By the numerical experiments we verify the validity of the algorithm, and comparethe effectiveness and the complexity of the algorithm for different modelselection methods.