The Effect Of Attribute's Sensitivity In Decision Tree Induction

Decision tree induction has been applied to the area of automatic knowledge acquisition, which learned from a set of cases to generate a decision tree. ID3 is the typical decision tree induction algorithm. It uses the information entropy as heuristic to build a crisp decision tree, which is based on assumption that the attribute's values and classification are crisp all. But, fuzzy decision tree induction is an important way for learning from examples with fuzzy representation. It is a special case of fuzzy decision tree induction extracting rules from the data, which have symbol features and crisp classes. Because building optimal fuzzy decision tree is NP-hard, it is necessary to study the heuristics. In this article, we compare two heuristics, i.e. FuzzyID3 and Min-Ambiguity algorithms, by using symbol and crisp data. We found ID3 algorithm is better than Min-Ambiguity in training accuracy, testing accuracy and size of tree by experimental and theoretical analysis. Meanwhile, we propose a new heuristic. So far, there have been many heuristic algorithms for building decision trees. Most are based on the entropy of information or vagueness, such as ID3, Min-Ambiguity and their variation. The new heuristic proposed in this article is based on the importance of attribute contributing to the classification. We do some primarily research about the sensitivity of attributes in decision tree induction. Usually, when generating decision tree the sensitive attribute is selected, but the insensitive attribute is ignored. By conducting experiments on several databases and comparing from different aspects, we find selecting the two kinds of attribute as extended attribute have each advantages and disadvantages.