Selective Ensemble Learning Algorithm Based On Pairwise Diversity Measures

Ensemble learning is a kind of machine learning paradigm. Through using multiple learners to solve one problem, it can effectively improve the generalization ability of learning systems. Therefore it becomes an international hot topic in the field of machine learning. At present, ensemble learning has been successfully applied in many fields, such as planet exploration, seismic wave analysis, text categorization, biological feature recognition, remote sensing information processing and computer aided medical diagnostics. However, ensemble learning technique is still immature, its theory has many unsolved problems, and its applications need to be further expanded and improved.It is generally recognized that the key in ensemble learning is to effectively generate individual learners with strong generalization ability and great diversity. However, the question still remains open about how to effectively measure and acquire and utilize this diversity. Selective ensemble by using parts of the individual learners in ensemble could be better in performance than that by using all individual learners in ensemble. So it has become an important research topic of ensemble learning. A better selection strategy and improvement of the speed of algorithm need more researches.In this thesis on ensemble learning, we introduce its related concepts, theoretical basis, constitution and two classical methods of ensemble learning algorithm (Boosting and Bagging). We apply ensemble learning to face recognition and compare with several common classifiers in face recognition. Then we do an in-depth study on selective ensemble. First, we introduce its essential concept and theoretical basis. Second, we introduce the algorithm of Genetic Algorithm based Selected Ensemble (GASEN) and the development of selective ensemble. Finally, based on pairwise diversity measures, a new algorithm (Pairwise Diversity Measures based Selective ENsemble, PDMSEN) and its improved algorithm (PDMSEN-b) are proposed. The main contributions of this dissertation are summarized as follows:(1) We apply ensemble learning (Boosting RBF neural network) to face recognition and compare with several kinds of classifier widely-used in face recognition. Experimental results demonstrate that ensemble learning and SVM methods have achieved relatively better performance and are most suitable for feature classifier in face recognition. The research provides a reference for selecting an appropriate classifier in face recognition.(2) In order to improve the diversity and accuracy of learners, Pairwise Diversity Measures based Selective Ensemble (PDMSEN) is proposed. Furthermore, a new method (PDMSEN-b) is presented to improve the speed of the algorithm which also supports parallel computing. At last, through applying BP neural networks as base learners, we test on selected UCI database and compare with Bagging and GASEN (Genetic Algorithm based Selected Ensemble) algorithms. Experimental results demonstrate that the learning speed of the proposed algorithm (PDMSEN-b) is superior to that of the GASEN algorithm in the same learning performance.