Converting a trained neural network to a decision tree DecText - decision tree extractor

Neural Networks are very successful in acquiring hidden knowledge in datasets. Their most important weakness is that the knowledge they acquired is represented in a form not very understandable to humans. In real life applications understandability of the knowledge acquired by a machine learning system is important.; In this thesis, we address the understandability problem of neural networks by converting the knowledge acquired by the neural network into a more understandable form. We do this by extracting a classical decision tree from neural network. We tested the new method on neural networks trained by using real life datasets. We showed that the decision trees extracted have high fidelity to the neural networks they are extracted from. Fidelity measures the closeness of decision tree's outputs to neural network's outputs.; We introduced four new splitting techniques to make DecText increase the fidelity of the extracted trees. We showed that the new methods are effective in extracting high fidelity trees. We also introduced a new discretization technique to make DecText be able to handle continuous features. New discretization technique can be used in other induction algorithms if a neural network is trained by using the dataset. We also introduced a new technique to prune the trees extracted. New pruning technique tries to find the simplest tree with the highest fidelity.; Another contribution of this thesis is a new feature subset selection technique. Real world classification applications usually have many features. This increases the complexity of the classification task. Choosing a subset of the features may increase accuracy and reduce complexity of the acquired knowledge. We tested the new technique on real world and artificial datasets and compared its results with existing methods. We showed that the new method chooses good subsets by searching fewer states than the existing methods. In the new method, we first sort the features according to their relevance and test the subsets formed by the most relevant features to find a starting subset for searching the subset space. We show that this technique speeds up the search considerably for most of the problem domains.