Research On Ensemble Extreme Learning Machine Algorithm Based On Multimodal And Multi-objective Differential Evolution

As a class of machine learning methods with powerful generalization performance and strong stability,ensemble learning has been applied in many real-world scenarios.However,in the process of constructing ensemble models,there are two tasks that need to be addressed: one is how to generate diverse and strong predictive individual learners;the other is how to combine the trained individual models into the final ensemble model.The generated individual models need to meet the two goals of "good and different" that are mutually restrictive and conflicting.Improving the performance of one of the goals often sacrifices the performance of the other.At present,there are many studies using multi-objective evolutionary algorithms to optimize these two conflicting targets,but it is difficult for multi-objective optimization algorithms to find all the optimal solutions to train more optimal individual models,so the performance of ensemble is reduced.Therefore,based on the multimodal multi-objective optimization algorithm and the extreme learning machine as the individual learner,this thesis studies the problems of ensemble learning in constructing individual models and combining predictions of individual models.The specific contents of the research are as follows:(1)In view of the problem that the optimal individual model will be lost when the multi-objective evolutionary algorithms optimize the generation of individual model pools,and the multi-modal multi-objective optimization algorithm can find all the optimal solutions of the optimization problem,this thesis proposes an individual model pool construction algorithm based on multimodal multi-objective differential evolution.First,the population is initialized,and the combination of features and hyperparameter of the extreme learning machine is encoded as individual decision variables of the optimization algorithm.Next,a group of extreme learning machines are trained and the fitness is calculated,where the prediction error of the individual model and the size of feature scale of the training data are used as evaluation indexes;then,the evolution operators are used to update the population;finally,iterate until the termination condition is satisfied,and the optimal individual model pool in the population at this time is obtained.Ten data sets are selected from the UCI database to test the algorithm.Compared with the traditional multi-objective optimization ensemble individual generation methods,the results prove that the proposed algorithm can find more and better Pareto solutions,thereby training more the best individual models.(2)For the combination of individual models in ensemble learning,this thesis proposes an ensemble learning method based on Pareto solutions.First of all,it is analyzed that suitable ensemble selections are beneficial to reduce the complexity of the ensemble model and will not reduce the generalization performance.Then,based on the optimization of the individual model pool generated by the multimodal multiobjective differential evolution algorithm,three ensemble selections are designed: 1)all Pareto ensemble;2)minimum error ensemble;3)complementary incremental ensemble.Next,the selected model predictions are combined into the final model output by majority voting.Finally,the experimental results show that compared with the existing ensemble learning methods and feature selection algorithms,the proposed algorithm has better generalization performance.(3)Considering that in real-life scenarios,the acquired data typically occurs class imbalance,this thesis adjusts and improves the proposed ensemble learning method based on multimodal multi-objective differential evolution.Taking G-means of individual models and the size of the feature scale of the training data as optimization goals,and the weighted extreme learning machine replaces the extreme learning machine as individual learner.An ensemble weighted extreme learning machine algorithm based on multimodal multi-objective differential evolution is designed and used for the classification problem of imbalanced data sets.Ten data sets are selected from the KEEL database for testing the algorithm.Compared with the existing imbalanced data classification algorithms,the results proved that the proposed algorithm can obtain better prediction results.