A Study On Designing Interpretable And Comprehensible Neural Network Trees

There are mainly two approaches for machine learning. One is symbolic approach and another is non-symbolic approach. Decision tree (DT) is a typical model for symbolic learning, and neural network (NN) is the most popular model for non-symbolic learning. Generally speaking, symbolic approaches are suit for producing comprehensible rules, but not well for incremental learning. Non-symbolic approaches, on the other hand, are suit for on-line incremental learning, but cannot provide comprehensible rules. Neural network tree (NNTree) is a hybrid-learning model of neural network and decision tree, which is a DT with each non-terminal node being an expert NN. It may combine the advantages of both DT and NN. To make the NNTree model practically useful, one should propose an efficient algorithm for incremental learning, try to produce NNTrees as small as possible, and provide a method for on-line interpretation. The purpose of this dissertation is to design interpretable, comprehensible or self-interpretable NNTrees based on both multiple objective optimization algorithms and multiple template matching algorithms. The main contribution of this dissertation is as follows:Reviewing exited design algorithms for hybrid model of NN and DT.Researching the definition and evolutionary design method of NNTree, which is based on ANN topology and its leaning algorithms, definition of DT and its learning algorithms, and optimization algorithms based on genetic algorithm (GA). Proposing a GA based multiple objective optimization for designing interpretable and comprehensible NNTree with reference to achievement of the limited input NNTree. The proposed method can increase the partitioning ability of the non-terminal node, decrease the number of inputs and the number of hidden neurons by the way of optimizing these three objectives. The efficiency of the method is tested by 4 machine leaning databases from UCI.Proposing self-interpretable NNTree by multiple templates matching method. In this method the multiple templates are used instead of MLP in original NNTree, and the design of templates is via the distance optimization between input vector and network weights. The analysis indicates that this kind of NNTree is characterized by the self-interpretation and comprehension and with equal generalization and classification ability as original NNTree. The theoretic analysis and a lot of experimental results stated the efficiency of the method. Devoting to discussions and conclusions.