Heuristic Selective Ensemble Learning Algorithm Based On Clustering And Dynamic Updating

Machine learning has been a key technique in many applications.Machine learning accesses to useful information through the analysis of the collected data modeling.The ultimate goal of machine learning is prediction.Ensemble learning is one of the most important area in machine learning.It is a new machine learning paradigm,which can solve the same problem through multiple classifiers.Until now,ensemble learning techniques have been applied in many fields successfully,such as image processing in the medical field,phoneme recognition,text classification and so on.However,ensemble learning is facing severe challenges with the increasing huge amounts of data.In 2001,Mr.Zhou proposed the concept of "selective ensemble".We can get a classifier set that has better generalization ability and better performance via choosing a part of classifiers set.However,how to get a classifier subset from the amount classifiers set that has the best performance?The research on it is still not ideal.The paper proposes a heuristic selective ensemble learning algorithm for the optimization problem on selective ensemble learning algorithm.So as to improve the predictive ability and the precision of selective ensemble learning algorithm.The paper firstly introduces the related concepts and algorithm of ensemble learning and selective ensemble learning.Then further describe the detail of the HCIU algorithm ideas.The main contribution of this paper includes following points.1.We propose a selective strategy based on the characteristics which can select automatically.In the step of pretreatment,reducing data dimension by three important metrics of InfoGain,GainRation and Pearson's.It can select the optimal feature set automatically.2.Optimize the parameters of classifiers with multi-thread technique.We can obtain the best parameters of classifiers quickly.3.Put forward the cluster technique to the trained classifiers.It can either reduce memory space it takes or improve efficiency by eliminating similar base classifiers.4.Put forward an ensemble model of dynamic updating based on Simulated Annealing.Select the sub sequence set of classifiers based on hierarchical selection and dynamical information.It can solve the problem in the past for choosing classifier to ensemble learning inefficiently.5.Put forward the divide-and-conquer strategy.It is employed to reduce the time cost for ensemble voting.The big voting task can be divided recursively into small child task by dichotomy,then the tasks are executed in parallel and it would conquer the voting result.