Structure Optimization And Improvement Of High Order Neural Networks

Basic concepts and development of neural networks are introduced in this dissertation, and the structure and learning methods of neural networks are also introduced. Several common high order neural networks, and the principle and some research questions of PCA are described in details. For the "dimension disaster" problem in the high order neural networks, an algorithm based on PCA for removing redundant neurons is presented. The new algorithm for high order neural network is investigated by some numerical experiments. The numerical experiments for boolean mapping problem and UCI problems show that the new algorithm performances better than common BP algorithm in generalization capabilities and learning speed. In order to study how the numbers of pruned neurons effect the network, the numbers of pruned neurons increase one by one. The experiments show that proper numbers of pruned neurons have a good effect, but excessive numbers of pruned neurons can damage the structure of original network. Part high order neural network is introduced to improve the application of high order neuron network. Part high order neural network is investigated by some numerical experiments. The numerical experiment for Gabor function problem show Part high order neural network performs better than high order neural network in generalization capabilities. The numerical experiment for N-bit parity problem show Part order-high neural network performs better than high order neural network in learning accuracy and speed, but in the two-spiral problem, the part high order neural network is highly unsatisfactory compared with higher order neural network.