Multi-level Selective Ensemble Learning Based On Multi-task Learning

The generalization ability of classification systems is the ability of algorithm to adapt to the new sample, how to effectively improve this ability is currently, in the study of machine learning, one of the most inportant challenges. The support vector machines and ensemble learning are now the two important approachs for improving this.Ensemble learning can significantly improve the generalization ability of learning systems through training a finite number of the base classifiers and then combing their results, however how to choose the base classifier to ensemble is a difficult problem. In this paper, we give an algorithm named Multi-level Selective Ensemble Learning Based on Multi-task Learning (MR-BMLSEN) which could deal with this problem well. The discussion to this topic is as follows:First, how to choose the base classifier for ensembling is a difficult problem in select ensemble learning; to deal with this we give a strategy which named Multi-level Selective Ensemble Learning.Second, prove the theorem, from the statistical point and geometric point to explain why it can work well?Third, study the theorem that named principle component analysis, explain why it can work well? Further show that multi-task learning, based on PCA, can improve generalization ability of classification systems with the ensemble learning.Fourth, on the WEKA platform, using UCI data sets, we make an experiment. We give a comparative analysis for MR-BMLSEN with the traditional machine learning algorithms J48, Adaboost and Bagging. Then we do confidence and effectiveness analysis on MR-BMLSEN. Experiments show that MR-BMLSEN can stably generate ensemble of decision trees with better generalization ability.