Research On Model Selection Of Support Vector Machine

To improve the generalization performance of support vector classification, the Gaussian kernel with multiple widths is proposed to emphasize the different contributions of features to classification. With this kernel, the related model selection scheme is designed which can automatically tune multiple parameters for support vector machines by minimizing the radius margin bound on error expectation of L1-SVM. The ideas are validated via experiments.In SVM classification, Gaussian kernel is firstly selected with high performance and its width defines the generalization scale. However, Gaussian kernel is not well adaptive everywhere in the pattern space if the patterns are unevenly distributed. That is, the over-fitting learning will appear in dense areas and otherwise under-fitting learning in sparse areas, and consequently, both will cause local risks. To reduce such local risks, a secondary kernel with global character is introduced which can enhance globality for Gaussian kernel. Gaussian kernel with local character is used as primary kernel. The constructed kernel is called the primary-secondary kernel (PSK). The positive definiteness of the secondary kernel with given constraints is proved by virtue of power series. Besides, two-stage model selection based on genetic algorithms is proposed to tune hyper-parameters for PSK. The algorithms first optimize the parameters of the primary kernel and then the ones of the secondary kernel. The objective function to be minimized is the radius-margin bound of L1-SVM. Experiments demonstrate that PSK performs better than Gaussian kernel and also validate the efficiency of the proposed algorithms.