An Evaluation Method Of Multi-class Classification BO-AUC

Classification is a very important data mining techniques. Classification performance was first measured by the accuracy, the research and experimental results show that the method of AUC (the area under the Receiver Operating Characteristic curve) is superior to the traditional methods. However, the accuracy also have the limitations, mainly due to the AUC method is based on two types of classification, can directly assess the performance of second-class classification model, can not assess the multi-class classification model directly. The B-AUC method which based on the F-AUC method be propsed. B-AUC (Binary-AUC) have three disadvantages. Include: 1) in order to express the logical relationships between nodes, must be stored in the form of complete binary tree nodes, which will waste of storage space; 2) based on partial binary tree search efficiency is higher than the other one; 3) as the small class sample, there are large difference between the two class that may lead to unbalanced data, the results of the evaluation;Because of the disadvantages of the B-AUC method, we analysis and summarizes the ROC analysis and AUC methods, and propsed the BO-AUC (Optimal Binary AUC) assessment methods which based on the data structure class of complete binary tree. By Category combinations in order twenty-two, calculate the AUC values of two class which are in the order , or calculate the AUC values of two class which are not in the order ,then get the average of AUC value which is the last results. The BO-AUC assessment methods run in the platform called MBNC (Bayesian Networks Classifier using Matlab) . Using the BO-AUC method to experimental the data sets that have more than three classes,to calculate the different modal. Analysis the difference between the B-AUC method, the results show that the method can effectively assess and compare the performance of classifier.