Research Of Classification Methods Based On Evidence Theory

Because there are huge amounts of data and information that have been generated andstored so far, people are not just satisfied with querying and statistics. It is a moresophisticated demand to analysis data and information intelligently and automatically. Thetechnology of data mining and machine learning can help us extract potentially valuableinformation and knowledge from huge amounts of uncertainty and noisy data. Classificationmethod, as an important method of data mining, machine learning and pattern recognition,can help people to make prediction, analysis data and recognise unknown patterns.Evidence theory is a powerful tool for handling uncertain problems and representing theuncertain knowledge. Combining the framework of the evidence theory to the classificationprocedure can effectively improve the accuracy and capability of the classifier. Currently,there are two main aspect of research about combining the evidence theory with theclassification method: firstly, combining the framework of evidence theory to the procedure ofsingle classifiers to improve its performance; and secondly, applying the combination rule ofevidence theory to multi-classifier ensemble. In this paper, we study the classification methodbased on the framework of evidence theory thoroughly, our works and innovations are asfollows:(1) A new classifier called subspace local mean evidence classifier (SLMEC) ispresented. The method first calculates the local mean vectors of each class regarding itsdistance with the test sample as evidences. Then for obtaining enough evidences, SLMECaccumulate some evidences from randomly divided subspaces of feature space. For eachevidence, the basic belief assignment is computed according to the distance between localmean vectors and test sample. In the following all these evidences represented by basic beliefassignments are then pooled together by the Dempster’s rule, and finally SLMEC assigns theclass label to test sample based on the combined belief assignment. The novel work of thismethod lies in that the evidences are created not only in whole feature space but also insubspaces. Besides, we consider the local mean vector and its distance with the test sample asthe evidence. SLMEC not only can deal with noisy and unbalanced data but also has goodperformance on high dimensional data.(2) Based on the research of SLMEC, we present another new classifier called RandomSubspace Evidence Classifier (RSEC). RSEC still make use of the information in both thewhole feature space and subspaces as SLMEC, but it considers the local hyperplane and itsdistance with the test sample for each class as the evidences. The experiments in the datasets from UCI machine learning repository, artificial data and face image database illustrate thatRSEC often gives better performance when performing the classification task. Besides, likeSLMEC, RSEC can deal with unbalanced data and high dimensional data well. The goodperformance of RSEC validated that our idea of making use of information in both wholefeature space and subspace is effective.(3) On the research of multiple classifier ensemble based on the evidence theory, wepresent two kind of improved random forest algorithm, which make use of combinationmethod of evidence theory to replace the traditional voting method in classical random forestalgorithm. One of our methods is making use of the measure level output of decision tree asthe basic belief assignment for each class. In this method, the multiple classifier combinationby using Dempster’s rule of evidence theory can be very easy. For the other method we takeplace of voting method by the Rogova’s evidence theory based multi-classifier combinationmethod. It is showed by the experiment result that two improved random forest algorithm byusing evidence theory based combination method has better performance comparing theclassical voting based random forest.(4) Based on the foundational research of improved random forest by using evidencetheory based combination method, we proposed a new classification ensemble method calledEvidence Theory based Diversity Forest (EDF). In our method the base classifier is decisiontrees and the diversity of the base classifier is produced by the combination of randomsubspace method、Bagging and principal component analysis. On the stage of multipleclassifier combination method, our method is making use of the evidence theory basedcombination method. The experiment results on UCI machine learning repository, artificialdata and application on speech emotion recognition suggest that the proposed approach givesbetter performance with the compared ensemble learning method, such as random forest,decision forest and rotation forest.