Research On Improving Selective Ensemble Algorithm Based On Diversity Measures

The basic idea of the selective ensemble methods is to eliminate the classifiers with relatively poor performance and redundant classifiers due to similar classification performance in the classifier set,and retain some classifiers to participate in the final ensemble.Scholars roughly divide selective ensemble methods into four categories: selection,sorting,optimization and clustering.On the basis of in-depth research on selective ensemble algorithms,this thesis improves the selective ensemble algorithms from three different perspectives of selection,optimization and clustering,and solves the following three problems:(1)The dynamic selective ensemble algorithm generally uses local neighbor samples to evaluate the performance of the classifier,so that the prediction accuracy of the classifier model constructed by this method is not high;(2)In theory,the more classifiers participating in the ensemble,the better the classification performance,but with the increase of classifiers,the classifiers may generate several redundant individuals due to the similar classification performance.Therefore,it is necessary to research on how to reduce the scale of ensemble without decreasing the classification performance;(3)The selective ensemble algorithm based on K-means needs to predetermine the number of clusters and randomly select the initial cluster centers,which makes the method have certain limitations.Based on the above problems,the main work of this thesis is as follows:(1)A dynamic selection and cyclic ensemble of the classifiers algorithm based on featureweighted K-nearest neighbors is proposed.The new method uses support vector machine to measure the importance weight of features,and according to the different test samples,finds more better neighboring samples through the feature-weighted K-nearest neighbor algorithm,calculates the prediction accuracy of multiple classifiers for neighboring samples,selects the classifiers with higher classification accuracy according to the set misjudgment tolerance threshold,and then uses the Kappa diversity measure to select the classifiers with greater diversity,which realizes the cyclic ensemble of the system.The experimental results show that the new algorithm has higher classification accuracy than the common dynamic selective ensemble algorithm.(2)A heuristic ensemble selection method combined with diversity measures is proposed.Firstly,the method converts the diversity matrix between the classifiers to an adjacency matrix,which is represented in the form of an undirected graph.Secondly,the graph coloring algorithm is used to realize the classifier grouping,and then the classifier with the highest accuracy rate is selected from each grouping to form a candidate classifier set.Finally,an improved simulated annealing algorithm is used to select the best subset of classifiers from the set of candidate classifiers to achieve the ensemble prediction.The experimental results show that the new algorithm has higher classification accuracy than the traditional ensemble learning algorithms.(3)A selective ensemble algorithm based on adaptive clustering is proposed.The method uses a confusion matrix as a tool to measure diversity among classifiers.the method of maximum and minimum distance is used to determine the cluster center,which solves the problem that the selective ensemble algorithm based on K-means randomly selects the cluster center.The number of clusters to be divided is automatically determined according to the change rate of the sum of squared error,which solves the problem that the selective ensemble algorithm based on K-means needs to predetermine the number of clusters.The experimental results show that the proposed algorithm has better classification performance compared with the selective ensemble algorithm based on K-means.