| In the classification of high-dimensional data,in order to select the smallest feature subset that can explain the target well,it is necessary to find the relevant features of the prediction target and remove the redundant information by discovering the correlation between features.Existing methods usually consider the relationship between features from the perspective of the correlation between features,while ignoring the complementariness of predictive capabilities between features.This paper first introduces the filter feature selection algorithms based on information entropy criterion,analyzes the advantages and disadvantages of typical algorithms,and then gives the definition and calculation of feature complementariness based on information entropy.We apply the idea of feature complementariness to decision tree and propose a variable complementariness calculation based on the decision tree.Extending it to random forest and proposing a complementariness calculation based on random forest.Two feature selection algorithms are proposed based on random forest variable complementariness.Based on the definition of information entropy,the Maximum-Relevance and Maximum-Complementariness(MRMC)feature selection criterion is proposed to evaluate features from two aspects:local relevance between features and target and complementariness between features.A method to quickly calculate feature relevance and complementariness between features through random forest is used.Comparative experiments on 16 standard data sets show that when selecting a small feature subset,the MRMC based feature selection algorithm is superior to several benchmark filter feature selection algorithms CMIM,MRMR,DISR,JMI,JMIM and MRI in terms of classification performance and time efficiency;it is superior to random forest feature selection algorithm in terms of classification performance.Two maximum relevance maximum complementariness feature selection algorithms based on spectral clustering,Maximal information coefficient-Variable Complementariness Measure(Mic-VCM)and Gini-Variable Complementariness Measure(Gini-VCM)are proposed.In order to maximize the feature relevance and complementariness in the obtained subset,the complementary matrix is obtained based on random forest,and the feature complementary matrix is converted to be the input of the spectral clustering algorithm,thereby obtaining k feature clusters,in each cluster the feature with the highest relevance is selected,and a subset containing k features is obtained and output.Comparative experiments on 16 standard data sets show that the Mic-VCM and Gini-VCM algorithms are superior to several benchmark algorithms in terms of classification performance and so on. |
