Research On Ensemle Classifiers Method For The High-deminsional Data

The ensemble classifiers method is one of the main domains in pattern recognitionand data mining, it has been developing rapidly in decades, since the classificationperformance is always superior to that of other single classifier method. In the researchof ensemble classifiers method, both the accuracy of base classifier and the diversitybetween them are two main factors, it leads to a better classification performance withhigher individual accuracy and lower diversity in ensemble classifiers system. Inhigh-dimensional data classification task, the accuracy and diversity can decrease bytraditional ensemle classifiers because of the reduncdancy and aparse of data features,so that the classification performance can be worse. Therefore, this paper proposes anew ensemble classifiers method based on ased on Rotation Subspace and LocalityPreserving Prejection (RSLPP). First, the original data set is projected in differentrotation subspaces (RS) to obtain differrent feature sub-sets to improve the diversitybetween base classifiers. Second, these sub-sets are prejcted by Locality PreservingPrejction (LPP) to train the base classifiers, so that the accuracies of base classifiers areguarateed. At last, the majority voting method is used to combing the output of eachbase classifier. The main research works in our article are as follows.①In the base of research of the background and development status of theensemble classifiers method, the article makes an in-depth study of basic theory of itand some classic ensemble methods, such as Bagging, Boosting, Random Forest andRotation Forest, and Rotation and focuses on Forest algrithem.②In our article, the exsiting ensemble classifiers methods are used forhigh-dimensional data classification task, the experimental results show that the accurayof base classifier decreases because of the reduncdancy and aparse of data features, andthe diversity beteween each base classifier declines with increasing in feature dimension.Therefore, the Rotation Subspace (RS) is proposed in our paper. First, the feature set issplited into several sub-blocks randomly. Second, one rotation subsapce is combinedwith the Eigenvectors in all feautre sub-blocks by the Random Sampling method, anddifferent rotation subspaces can be obtained in numerous iterations. At last, the originaldata set are prejected in these rotation subspces to obtain different feature sub-sets. Inthis way the diversity can be increased.③In process of constructing feature sub-sets, parts of feature with no use are remained in each sub-block to increase the diversity beteween each base classifier. Inorder to reduce these invalid feature from angle of feature sub-set, Locality PreservingPrejection (LPP) is utilized to do feature extraction in each feature sub-set, then the baseclassifier can be trained by each processed feature sub-set, so that the accuray of eachbase classifier can be improved. Finally, the outputs of all base classifiers are combinedto get the classification result of our proposed method by majority voting. In summary, anew ensemble classifiers method for high-dimensional data based on Rotation Subspaceand Locality Preserving Prejection is proposed.④In order to prove the validity of our proposed method for high-dimensionalclassification task, several data sets are collected (including seven data sets from UCIrepository and two face databases). Experiments are conducted to compare with otherexsiting classification methods, the recognition accuracy, bias-variance and kappa-errordiagram are have been introduced to evaluate the performance of each method, Theexperimental results have verified the effectiveness of the proposed method is effectivefor dimensional data classification task.