Study Of Ensemble Learning Based On Selective Strategy

Ensembles of learnt models constitute one of the main current directions in machine learning. Ensemble allows us to achieve higher generalization performances and tackle the problem of over fitting by fusing the results of many learning machine in some way. In result years, as machine learning has been widely used in data mining and analysis, a number of theories of ensemble learning have been successfully applied into image recognition, voice recognition and the classification of seismic wave and so on. The development of selective ensemble is based on the ensemble learning.In this paper, we primly study the ensemble learning methods of selective strategies. The contents of study include the following respects. First, some base models are respectively trained by the algorithm of ID3 and BP Neural Network. When using the algorithm of ID3 trains base models, the dataset need to be discretization, and when using BP Neural Network, the dataset need to be normalized. So there is necessary to study the dataset and analysis ensemble performance. Second, we study the selective strategies of Hill Climbing, Forward Sequential Selection and Backward Sequential Selection. The experimental results show that these selective strategies are effective; meanwhile, the results of altering the parameters of fitness function show that diversity influences on the ensemble performance. The results also prove the effective of selective ensemble by comparing the ensemble performance of all base models. Third, we also study the selective ensemble from the view of cluster technology. Here the cluster means that we can cluster some base models together. In this paper, cluster technologies include k-means and hierarchical clustering. In clustering technology, the methods of selecting ensemble models include four strategies, that is, selecting center object as ensemble model, randomly selecting one object as ensemble models from every cluster, randomly selecting two and three objects as ensemble models from every cluster, and finally measuring the diversity of ensemble models. The methods of measuring diversity include fail/no-fail, double fault and correlation coefficient. Fourth, the results of generalization error are analyzed. Comparing and analyzing the experimental results, validating the effectiveness of selecting center object as ensemble models and the effectiveness of selective ensemble. Consequently the results show that selective strategies can improve the generalization performance of ensemble learning. In future, selective ensemble can be fund usefully.