Research On Selection Of Kernel Functions And Key Parameters In Support Vector Machine

Based on Statistical Learning Theory, Support Vector Machine(SVM) is a new machine learning method developed in the 1990 s, and has been widely used in the field of pattern recognition, regression estimation and density estimation. As a crucial part of this method, the choice among a wide range of kernel functions and the tuning of related kernel parameters have a big influence on the behavior of the method. This problem has certainly attracted many scholars' research interest in this field.At the start of this thesis, the theory of Support Vector Machine is introduced, and the related Statistical Learning Theory is summarized followed by the analysis of the decisive effect of Statistical Learning Theory on the sound performances of Support Vector Machine. Then this thesis introduces the theory of kernel function, and makes a summary of certain properties of common used kernel functions, namely Radial Basis Function(RBF) kernel, polynomial kernel and Sigmoid kernel, providing a reference for the problem of kernel type selection. When it comes to the problem of how to choose kernel function type and tune corresponding kernel parameters, this thesis mainly focus on methods based on class separability measure and proposed a new class separability measure called Expected Squared Distance Ratio(ESDR). Then the properties of ESDR are analyzed in details, in comparison with other two class separability measures, showing ESDR's advantages in theory. Taking specific forms of data distribution into account, the expression of ESDR in the case of Gaussian data with RBF kernel is derived, on the basis of which the corresponding problem of kernel parameter selection is analyzed. At last, several real world datasets and Gaussian data are introduced in the RBF-SVM classification experiments. The experimental results on real world datasets show that compared with the other two class separability measures, ESDR can better quantify the class separability of data in the corresponding feature space of some kernel with some kernel parameters, and ESDR-based method can lead to better kernel parameters, and is far more efficient than the traditional grid search method. The experimental results on Gaussian data reveal that, ESDR can be employed to study kernel parameter tunning problem under specific data distribution with specific kernel, thus has great value in theoretical research within this field. All the experimental results of this thesis show that, ESDR is an excellent method that can be applied to solve the problems of kernel function selection and key parameter selection for support vector machines.