Approach And Application Research Of Tree-Structured Hybrid Learning Models

Tree-structured hybrid learning models have been introduced by a number of authors in recent years. It combines the advantages of both symbolic and non-symbolic models to solve tough problems and indicates great potential application in the fields of biomedicine, information security, fault diagnosis and facial expression analysis. The research of this paper is focused on two types of hybrid learning models named neural network tree (NNT) and support vector machine tree (SVMT). They are hybrid decision trees in which each internal node is embedded with modular expert neural network or support vector machine. The main research contents and contributions are listed as follows:1. An interpretable neural network tree based on self-organized feature learning (SFL-NNT) is presented. With the assumption that the inputs are all binary numbers, the interpretation of an NNT trained on continuous features may be too complex to implement. To solve this problem, we propose an interpretable NNT learning approach through self-organized learning of continuous features. Experiments show that the recognition accuracy of SFL-NNT is competitive to that of NNT. Further, the SFL-NNT can reduce the spatial computational complexity for interpretation greatly.2. SFL-NNT is applied to intrusion detection problem. SFL-NNT constructed on KDD intrusion data set reaches satisfying training and test detection accuracy. Furthermore, the learned model contains the understandable information about those features of critical importance for detection.3. A novel hybrid-learning model named confusion-cross-based support vector machine tree (CSVMT) is proposed. The problems associated with complex two-class pattern recognition problems are firstly addressed. The construction of a CSVMT model is implemented by embedding SVMs in the internal nodes of a binary tree, in which two training subsets assigned to two internal sibling nodes perform confusion cross. A simplified heuristic method is introduced to extend binary CSVMT to a multi-classification one. Experimental results demonstrated on two-class complex distribution problem and databases taken from the machine-learning repository of UCI show that the proposed approach is...