Researches And Application On The Structure Optimization Of The BP Neural Networks

Based on the problem of the optimal structure in BP neural networks (that is, the number of the hidden layer and the hidden neurons are difficult to determine) and the deficiencies of the BP algorithm, we resolve these problems from the pruning method and genetic algorithm in this thesis. And a fast OBS pruning algorithm based on pseudo-entropy of weights and BP neural networks structural optimization based on hierarchical genetic algorithm are proposed. MATLAB is as simulation platform to realize the function approximation of the Hermit polynomial and the nonlinear function, as well as a simple pattern classification problem, which verifies the corresponding algorithm.The main research contents are concluded as follows:i. Learning and researching the BP algorithm, based on its deficiencies, the pseudo-entropy of weights is as penalty-term introduced into the objective function to train the networks, which to make the network constrains the distribution of weights during training automatically and weights of the network tend to uniform distribution.ii. A fast OBS pruning algorithm based on pseudo-entropy of weights is proposed to resolve the problems of the number of hidden neurons which is difficult to be determined in BP neural networks, the OBS pruning algorithm is improved, and the speed of pruning is improved greatly. The results of experiments show that the structure of network has been simplified; the generalization capability of network and the effective of pruning have been improved greatly by using the algorithm mentioned above.iii. BP neural network structure optimization method based on the hierarchical genetic algorithm is presented, which using the hierarchical structures of the hierarchical genetic algorithm to optimize the networks'structure and weights respectively, and then using the BP algorithm to further adjust the network's weights with fixed network structure after the evolution, in order to achieve the network structure and weights'optimization. The results of experiments show that the structure of network has been simplified and the convergence speed can be greatly increased.iv. MATLAB was as the simulation platform to realize the function approximation of the Hermit polynomial and the non-liner function, as well as a simple pattern classification problem applying the above-mentioned method. By analyzing the advantages and disadvantages of each method and comparing the performance of the various parameters to verify the effectiveness of the algorithms.