Support Vector Regression Based On Penalized Splines

Support vector machine(SVM)is a supervised learning method to deal with classification and regression problems in statistical learning.It is well studied and applied in many cases because of its nice generalization ability.The kernel trick helps SVM to cope with the nonlinear problems.Another well-known tool to solve nonlinear problems is the penalized spline,where the regression function is fitted by the spline basis based on the sample observa-tions.It is well worked to capture the nonlinear characters between response and variables.In this paper,firstly,a new local penalty spline regression method is intro-duced to solve the problem of local heterogeneity of data.Bigger penalizations are given to the regions where data have more volatility and smaller penaliza-tions are imposed on regions where data are more stationary.This data driven skill well improves the adaptability of model.Secondly,SVM based on penal-ized splines is given,where the property of low rank of penalized splines can significantly reduce the computational complexity of SVM.Furthermore,an adaptive SVM is proposed to reduce the number of hyper parameters.Com-pared with the SVM based on penalized splines,this method obviously reduces the number of hyper parameters and outperforms the former in generalization.