Research On Hybrid Neural Networks And Diversified Ensemble Methods Based On ICBP Model

Our Improved Circular Back-Propagation (ICBP) network possesses the same topological configuration as standard Back-Propagation (BP) and Circular Back-Propagation (CBP) network proposed by S. Ridolla. Compared to BP network, the advantage of ICBP is: it has an extra quadratic input node, just as CBP does. While compared to CBP network, the improvements of ICBP mainly are: Firstly, the calculation of ICBP extra input item is different from that of CBP. As a result, ICBP network possesses anisotropic characteristic, breaking through CBP's limitation for the ability of representing isotropy only. Secondly, the values setting method of connection weights between the extra input node and all the hidden nodes in ICBP network is different from that in CBP, i.e. directly set these weights as a common constant. Our research validates that ICBP possesses stronger generalization and adaptation capability than BP and CBP. And it has better representation ability as well. It is a more general neural network model. Besides, experimental results on time series benchmark datasets and the data set of daily life water consumed quantity have proved that ICBP has better capabilities of prediction and approximation than BP and CBP.Although ICBP model achieves several ameliorations to BP and CBP, we find in our research that, it still can not successfully fulfill classification tasks of highly nonlinear data, e.g., to classify dates to the day of the week on which they fall, despite that its performance is apparently superior to BP and CBP. Moreover, we find throught experiments that even BP-SOM can not solve this difficult problem satisfactorily, either. These discoveries inspire our thoughts to improve BP-SOM architecture and our ICBP model further. Aiming at those common problems confronted with by BP, CBP and ICBP networks and starting with their limitations in generalization, this paper investigates, systematically and deeply, the technology of building hybrid neural network architecture and diversified neural network ensemble system based on ICBP network model. The research work of this dissertation consists of two parts: (1) Based on ICBP (Improved Circular Back-Propagation) model and under the architecture of BP-SOM, we investigate the hybrid neural network combination method, including ICBP-SOM and ICBP-KSOM; (2) Also based on ICBP model and integrating DCS (Dynamic Classifier Selection) algorithm, we build DCS-ICBP ensemble system, exploring the relationship of this explicitly constructed ensemble system and its diversity. The main contributions of this dissertation are summarized as follows:1. We realize two steps of alterations towards BP-SOM architecture in total. The first one is: displace the MFN (Multi-layered Feedforward Network) module in BP-SOM with ICBP network, obtaining ICBP-SOM. This step of alteration gains two aspects of benefits: namely, simultaneous ameliorations to the performance of both ICBP network and BP-SOM. Simulation results on six benchmark pattern classification tasks validate that ICBP-SOM acquires improvements to both ICBP and BP-SOM to a certain extent, accordingly approving the rationality and validity of this step of alteration.2. We introduce KSOM into BP-SOM architecture, and combine it with ICBP network using the similar combinatorial method. As a result, we gain a new hybrid neural network model ICBP-KSOM, and realize the second step of alteration to BP-SOM. The non-Euclidean distance measure of KSOM is beneficial for ICBP-KSOM to deal with non-Euclidean structure in data, while the high robustness of KSOM also advances the robustness of the whole ICBP-KSOM against noise and outliers. Consequently, this kind of combination upgrades the capabilities of both models. The results of benchmark classification experiments and t-tests validate the superiorities of this new combinatorial type of neural network architecture in both classification capability and generalization performance.3. We select five anisotropic ICBP networks (i.e., ICBPall+1, ICBPall-1, ICBP+1o-1e, ICBP+1fh-1lh, Random-ICBP) with five special assignments to the weights values between the extra input node and all the hidden nodes, and adopting the DCS algorithm based on DANN (Discriminant Adaptive Nearest Neighbor) measurement put forward by Didaci and Giacinto, we integrate those five anisotropic ICBP models to build a dynamic ICBP classifiers selection system (DCS-ICBP). This type of explicit and direct means of constructing diversified ensemble system is a meaningful innovation. To build diversity directly upon network structure is impossible for the popular BP network. Finally, DCS-ICBP system presents good classification results in experiments, putting up rather strong generalization capability.