Research And Application Of Pareto Ensemble Pruning

Ensemble learning is one of the widely used machine learning techniques,which usually requires training a lot of base learners.The existing applications and theoretical studies have shown that deleting some redundant or poor base learners of the ensemble system can effectively improve the generalization performance of ensemble learning.Pareto Ensemble Learning(PEP)[1] is a kind of algorithm that can regard the generalization performance and the ensemble scale which means the number of base learners as a double objective.In order to further improve the classification performance of Pareto ensemble pruning,this paper has done the research.And the research content mainly includes the following two parts,In the first part,the PEP algorithm only considers the precision and ensemble scale of base classifiers,while ignoring the difference between the classifiers,which leads to the classifiers be more similar.A new Pareto Ensemble Pruning algorithm called Pareto Ensemble Learning with Diversity(PEPD)is proposed,which integrates the diversity and precision of the classifier into the first optimization goal,and the ensemble scale as the second optimization goal.Thus multi-objective optimization have been achieved.In the second part,the data division part of Min-Max Modular Neural Network(M3)method is combined with the PEP algorithm during the processing of the imbalanced data set,which called the Application of Pareto Ensemble Pruning in Modular Network,referred to as APEPM.Different from traditional method of dealing with unbalanced data,it adopts the Min-Max Modular partitioning method,which divides unbalanced data into relatively balanced data subblocks.And then an optimal sub-classifier is obtained according to the subset search method of Pareto ensemble pruning.The experimental results show that the PEPD method can obtain higher performance in most cases,and the enhancement is due to diversity's combination when PEPD and PEP have the similarity number of base learners.And the APEPM method for unbalanced data processing can achieve better classification performance than traditional ensemble pruning method.