Research On Sensitivity Analysis Based Methods For Optimization Of Neural Network Architecture

Neural Network has been widely applied in various intelligent information processing fields, such as pattern recognition, signal processing and image processing. The performance of neural network is determined mainly by learning algorithm and the architecture of it. So the optimization of neural network architecture is important to neural network research. Neural network pruning, which eliminates least important nodes or weights, is an important method of optimizing neural network architecture. Sensitivity analysis can define the relationship between nodes or weights in network, and so provides theoretical foundation for neural network pruning.In this thesis, the further research into the sensitivity analysis based methods for optimization of neural network architecture is done. The results obtained are as follows:(1) The methods of optimization of neural network architecture are reviewed and the advantages and disadvantages of existing pruning methods are summarized.(2) The applications of sensitivity analysis to sensitivity analysis based neural network pruning methods are discussed. The feasibility of applying the Mutual Information Index to neural network pruning is analyzed.(3) The method of identifying and ordering important nodes according to the Mutual Information Index is presented. Furthermore, the method of input node pruning based on Mutual Information Index is developed. Experiments show that this method can identify important input attributes.(4) The method of hidden node pruning based on Mutual Information Index is proposed. Through this method the number of hidden units is determined. Then this method is applied to solving function approximation and pattern classification problems, and the effectiveness of the method is proved.Finally, the research work involved in the thesis is summarized and the future developments in optimization of neural network architecture are forecast.