Research On Convergence Of Learning Algorithm For Two Kinds Of Neural Networks

The feedforward neural network is one of the most widely used neural network models in applications. The fuzzy perceptron and the Ridge Polynomial neural network are such two kinds of neural networks. The major work of this paper is to study the learning method for those two kinds of neural networks, and the convergence of the learning algorithm is discussed. The main work is organized as follows:The asynchronous gradient method was applied to train the Ridge Polynomial neural network. By analyzing the error function and learning algorithm, the monotonicity theorem of the error function was proposed. Moreover, on the basis of the proved monotonicity theorem, the convergence of the asynchronous gradient method was analyzed, further the convergence theorem was proved. To illustrate the theoretical finding, numerical experiments were carried out and the experimental results demonstrated the proposed theorems are valid.Slow convergence rate and easily falling into the local minimum are the drawbacks of the traditional gradient algorithm, the momentum is one of the effective methods to solve the problem. In this thesis, momentum was introduced into the asynchronous gradient method to improve the convergence efficiency. The monotonicity and convergence of the improved asynchronous gradient method with momentum for Ridge Polynomial neural network were studied. Finally, a supporting experiment was also given and the simulated experimental results indicated that the improved algorithm is efficient.A learning algorithm was proposed for the fuzzy perceptron with max-product composition, and the structure of this fuzzy perceptron is similar to the conventional linear perceptron. The sufficient and necessary conditions for training patterns to be fuzzily separable were presented. This article proved that the proposed learning algorithm has the properties of finite convergence when the training patterns are fuzzily separable. To fully support the theoretical finding, some numerical simulation experiments for the learning algorithm were provided.