Research Of Ensemble Learning

Ensemble learning, which uses multiple learners to solve the same problem, can significantly improve the generalization ability of learning system. It has become an important field of machine learning research in recent years. Although two classical ensemble learning methods--Boosting and Bagging--have been studied in a deep way, there has no uniform rule for the ensemble learning algorithm design. Therefore, to study ensemble learning with better performance from other perspectives is a trend and also necessary. As a special ensemble learning paradigm, Selective ensemble selects individual learners with great differences and better performance to involve in the ensemble. It achieves better performance than the ensemble with all the learners. So how to choose the individual learners to involve in the ensemble becomes a research focus.In this paper, a simple description of the relative theory about ensemble learning and a summary of algorithm & technology about ensemble learning are given. Then, two main works are finished in this paper as follows.Firstly, a new ensemble learning method based on pairwise constraint and subsets selection is given. There have two innovations in this algorithm. One is to introduce "pairwise constraint" emerged in semi-supervised clustering to the Bootstrap sampling of Bagging. The other is that a function which is to evaluate the category dispersion of a subset is defined. Selecting individual learner is achieved via selecting a subset with good category dispersion indirectly. In order to compare this method with Bagging in performance, we use the 10 UCI standard data sets in Matlab7.0 to verify their performance. The result proves that this method is superior to Bagging.Secondly, a novel selective ensemble learning method called mRMR-MISEN is constructed. Multi-information is used to select individual learners. The innovation of the algorithm is to extend "the most relevant minimum redundancy criterion" in feature selection to learner selection in ensemble learning. Therefore, the performance of individual learner is not only considered, the influence among all individual learners is also considered. This method and other compared algorithms such as MISEM, CMISEN are implemented in Matlab7.0 with 10 data sets. The result shows that the algorithm is better than the other two algorithms in prediction accuracy.