Super Gaussian Kernel Function Of Support Vector Classification Machine Parameter Range

Support Vector Machine, or SVM, which is firmly based on statistical learning theory, especially the Structural Risk Minimization principle therein, and highlights the generalization capacity of machine learning, is one of the most significant development in machine learning since Artificial Neural Network, or ANN. Investigation on SVM research literature exhibits that it is still one of the most active research area in machine learning after more than two decades’ extensive and profound research. Through extensive investigation and review on domestic and international literatures involving effective hyperparameter range of C Parameter Support Vector Classification, or C-SVC, the author concludes that all literatures involving hyperparameter range or intial value settings simply set them empirically or semi-empirically at most, which argues that no systematic research is reported that draws special attention on SVM effective hyperparameter range to the best of the author’s knowledge. Therefore, the author established the effective hyperparameter range of C-SVC with Gaussian kernels, or Radial Basis Function (RBF), as the research subject, and seeking methods that could obtain effective hyperparameter range dependent on SVM intrinsic and essential properties and relevant to datasets as the research goal.After a description on the research background and significance, a review on demestic and international research status, and an exposition on the basic theory and solution methods of SVM, the dissertation has executed integrated and extensive research on the planned goal, and conducted enormous basic and verification experiments. The achievements can be summarized as follows.1) Through detailed investigation and review on parameter tuning literatures and literatures that apply parameter tuning in C-SVC, an important conclusion is reached:the difficulty of SVM parameter tuning lies in that the SVM generalization error is a non-convex multiple minimum function with respect to its hyperparameters. GS k-CV error surface and contrast experiments on typical tuning results both confirm this fact. Curves exhibiting the variation of the number of support vectors with respect to parameters verifies that when lowered to certain value, all the minority samples become support vectors, and the number of support vectors in the majority samples is just equal to the number of minority samples. Curves showing the variation of generalization error with respect to parameters enable the author to argue that simple parameter tuning methods can be devised by invoking training on the whole dataset.2) By empirical analysis, a dataset dependent algorithm for determining effective range is devised, and an algorithm for determining effective C range is also devised. Experimental results shows the validity of the two algorithms and demonstrates that they can substantially eliminate the GS5-CV computation to1/6.3) A heuristic and dataset dependent method to determining effective range is devised by means of the asymptotic property of RBF. Based on the SMO algorithm, a lower limit Cparameter with respect to function is derived, which is dataset related and also irrelevant to concrete kernel functions. Experimental results shows that the methods as a whole can delineate the specific shape of RBF SVC effective hyperparameter range properly.4) The most innovative research achievement of this dissertation is that a systematic method has found that can delineate the specific shape of effective RBF SVC hyperparameter range tightly and perfectly. The method mainly includes the following major points:(1) Expresses the SVC optimization problem as a combinatorial optimization problem when C is lowered to certain limit and proposes a Greedy Search method to solving this combinatorial optimization problem and getting and b.(2) Expresses the margin bound of the mojority samples and the classification decision boundary as an linear equation system of variables and b when C gets to, and proposes to solve this equation system by LMS to get and b by applying the results of the above Greedy Search method. For RBF SVC, obtained from LMS is relevant to the hyperparameter, which means that a related function is obtained.(3) According to the property that RBF tends to0when gets sufficiently large, a property of which states that no longer changes with when gets higher than a certain threshold is inferred, which can be employed to determine the upper limit of the effective range.(4) According to the property that RBF tends to1when gets sufficiently small, another property of which states that asymptotically changes to a linear function of when gets lower than a certain threshold is inferred, which can be employed to determine the lower limit of the effective range by resorting to an empirical cut-off value of SVC classification margin. Experimental results demonstrates that the proposed method as a whole can delineate the specific shape of RBF SVC effective hyperparameter range tightly and perfectly.The above4research achievements mark sufficient job of the dissertation in investigation and review, analysis and derivation, algorithm design and experiments. The achievements also confirms that the planned research goal has been fully realized.