Research On Ensemble Regression Learning Based On Classifier Selection And Multiple Kernel Selection Under Least Squares Framework

Classification and regression techniques have been widely applied in many fields such as face recognition,satellite image recognition,and information security.Ensemble learning can effectively improve the classification and regression results for an individual model.However,most of the previous studies pay much attention on the accuracy of ensemble learners,and neglect the selection of differences between base learners.Therefore,the overall generalization performance of ensemble learners is still insufficient.At the same time,scholars have also ignored the use of kernel learning in ensemble learning.Thus,this paper has studied how to select appropriate base learners,and joined the kernel techniques in proposed method.Also our method has converted the classification and the regression problem into an ensemble regression model through a least-squares frame,so that the final ensemble model has better generalization performance and robustness on classification and regression fields.The main research contents of this thesis are as follows:The diversity selection strategy in classification ensemble learning under the least squares framework has been proposed.In view of the problem that the formers ignored the differences of the base learners,this paper selects base classifiers of ensemble model by combining accuracy and diversity.The least squares framework is used in this strategy to maximize the selection of the base classifiers.Meanwhile,the strategy aims at obtaining the minimum classification error through a regression framework in the whole process.It can adaptively assign the weights of the base classifiers.Experimental results show that the proposed method has better classification performance than other ensemble methods such as Random Forest,AdaBoost and EnsembleSVM.The multiple kernel selection strategy in regression ensemble learning under the least squares framework has been proposed.In order to improve the effectiveness of ensemble learning,this paper takes kernel technique in the process of selecting base learners,and applies ensemble learning to the regression problem.These base kernel regressors are co-regularized and co-optimized in the whole process.The problem of finding suitable kernel types and their parameters in base kernel regressors can be solved in multiple kernel models.The strategy is used to optimize the overall squared loss of multiple kernel base regressors in multiple Hilbert spaces under the least squares framework to obtain optimal ensemble results.Experimental experiments on artificial datasets,UCI regression and classification datasets,and handwritten digital datasets,show that proposed algorithm has better classification and regression results in comparison with other regression methods such as Ridge Regression,Support Vector Regression,and Gadient Boosting Regression.