Research On Structural Diversity Of Ensemble Learning

Ensemble learning is a notable category of machine learning,which constructs and combines multiple learners for one learning task,usually has a significantly superior generalization performance than a single learner.It has achieved great success in prac-tice.The diversity between learners is a crucial factor to the construction of ensemble methods.How to understand and measure ensemble diversity is a fundamental prob-lem,while still remains open.This thesis focuses on researches on structural diversity in ensemble learning,and achieves the following innovations:Firstly,the concept of structural diversity is proposed,which opens a new direc-tion for researches on ensemble diversity.Existing diversity measures consider only behavioural diversity,i.e.,how the classifiers behave when making predictions,ne-glecting the structural difference between learners.A structural diversity measure for decision tree is proposed and experiments demonstrate the effectiveness of structural diversity.Secondly,a new selective ensemble method named DRSE(Diversity Regularized Selective Ensemble)is proposed.The new method considers ensemble error and the two kinds of behavioural and structural diversities simutaneously,and is optimized by DCA algorithm.Experiments show that the method is superior to the previous selective ensemble methods.Thirdly,a method for estimating multi-information diversity based on junction tree is proposed.Multi-information diversity is a method for portraying ensemble di-versity based on information theory.In practice,it is difficult to estimate the high-order information.This thesis proposed an estimation based on junction tree.Experiments demonstrate the superiority of the proposed methods to existing ones under the same low-order approximation.