| In the past 20 years,kernel methods have been successfully applied to solve many machine learning and pattern recognition problems.As we all know,the performance of kernel methods highly depends on the choice of kernel parameters.Traditional model selection is implemented based on grid search,random search or manual search under the framework of cross-validation.However,these methods select a limited number of candidate kernel parameters in a discrete parameter space,and then train the model multiple times,which requires a high computational cost.Although researchers have proposed kernel path algorithms for selecting kernel parameters in a continuous parameter space,these methods still cannot guarantee the search for the optimal kernel parameter value in the entire parameter space.This research will further discuss the kernel method model selection algorithm for classification,regression and ordered regression problems.The main algorithms include kernel path algorithm,general kernel path algorithm and kernel error path algorithm.In this study,the following researches were conducted on the model selection problem of the kernel method:(1)For the ν-support vector machine,this paper proposes a corresponding kernel path algorithm,which avoids repeated training of the ν-support vector machine,and can tune the kernel parameters in a continuous space.More importantly,this paper analyzes the convergence of the proposed algorithm.Finally,experiments on multiple classification datasets verify that the proposed algorithm improves the classification accuracy of the ν-support vector machine.(2)For a variety of machine learning problems(such as standard support vector machine,LASSO,ordinal regression support vector machine,etc.),this paper first proposes a general parameterized quadratic programming problem,and then designs a general kernel path algorithm,which solves the problem of kernel parameter tuning in a continuous parameter space for a variety of learning algorithms.Finally,this paper gives an example of this general algorithm instantiated into a standard support vector machine.(3)Although the kernel path algorithm can realize the optimization of the kernel parameters in the continuous parameter space,it still cannot guarantee that the model with the minimum cross-validation error is selected.In order to solve this problem,this research proposes a kernel error path algorithm for classification and regression problems.The algorithm minimizes the cross-validation error to ensure that the global optimal model is selected.Finally,this paper analyzes the convergence of the proposed algorithm.The experimental results on a variety of datasets not only verify the effectiveness of the kernel path algorithm,but also show that the model combined with the core split path algorithm selection is better than models selected by the traditional algorithms. |
