Composite Parameters Selection For Support Vector Machines

Automatic tuning of (hyper)parameters is an essential ingredient and important process for learning and applying Support Vector Machines (SVM). The existing tuning methods choose (hyper)parameters and classifier separately in different iteration processes, and usually search exhaustively in parameter spaces. In this thesis, We propose and implement a new tuning algorithm that chooses (hyper)parameters and classifier for SVM simultaneously and search the parameter space efficiently with a deliberate initialization of a pair of starting points. First we derive an approximate but effective radius margin bound for soft margin SVM. Then we combine multiparam-eters of SVM into one vector, converting the two separate tuning processes into one optimization problem. Further we discusse the implementation issue about the new tuning algorithm, and that of choosing initial points for iteration. Finally we compare the new tuning algorithm with the gradient based method and cross validation on five benchmark data sets. The experimental results demonstrate that the new tuning algorithm is effective, and usually outperforms those classical tuning algorithms.