Model Combination Of SYM On Regularization Path

Model combination aims at improving the generalization performance and relia-bilities of learning systems. In previous model combination practices, base models ofmodel combination are obtained by using data sampling methods, which is computa-tionally complex; combination is performed on all available models without consider-ing the new input; little attention has been paid to the theoretical proof of consistenciesof model combination.To address these issues, we explore the model combination of support vector ma-chine (SVM) on regularization path. We prove the consistency of model combinationof SVM and propose a efective method for model combination of SVM.The main contributions of this dissertation are shown as following:1. We improve the regularization path algorithms for support vector classification(SVC) and support vector regression (SVR) using positive definite matrix. Byapplying the Cholesky decomposition of positive definite matrix, the improvedalgorithms can handle data set having duplicate points, nearly duplicate pointsand linearly dependent points efciently.2. We define the Lh-risk consistency, and prove that the model combination of SVCbased on regularization path is Lh-risk consistent. We define the L-risk consis-tency, and prove that the model combination of SVR based on regularization pathis L-risk consistent. The consistency results provide the mathematical founda-tion for the model combination of SVM on regularization path.3. We propose a three-steps strategy to construct the model set for model combi-nation of SVM on regularization path. The first step is to get an initial modelset from regularization path. The second step is to prune the model set toexclude models with poor performance from the initial model set by Occam’sWindow method, average of GACV or average of GCV. The last is to determinefinal model set by the input-dependent method, and perform the Beyesian combination.The proposed Bayesian combination model algorithms are theoretically sound andpractically efective.