Algorithm Design And Analysis For Conjugate Neural Networks Based On Generalized Armijo Search

Artificial neural network(ANN)is widely used in signal prediction,function approximation,automatic control,pattern recognition and so on.The single hidden layer feedforward neural network with universal approximation ability is the focus of the study of ANN.Back propagation algorithm based on steepest descent method is one of the popular algorithms for training single hidden layer feedforward neural network,but it has some disadvantages,such as slow convergence and long time-consuming.Based on the advantages of low memory and fast convergence,conjugate gradient method has become an effective algorithm for training neural networks.In recent years,researchers have proposed a real-valued neural network algorithm based on conjugate gradient method,which combines the single hidden layer real-valued feedforward neural network model.For unconstrained optimization problems,a three-terms conjugate gradient algorithm based on generalized Armijo step rule is proposed to speed up convergence.In this paper,for real-valued and complex-valued neural networks,the real-valued and fully complex-valued conjugate neural network algorithms based on generalized Armijo search are proposed respectively,and their convergence is proved in detail.Some numerical experiments are given to verify the rationality and effectiveness of the algorithm.The main contributions of this paper are as follows:1.A real-valued conjugate neural network algorithm based on generalized Armijo search method is proposed for single hidden layer real-valued BP neural network.On the one hand,the convergence speed of the algorithm is greatly improved by conjugate gradient method and generalized Armijo line search strategy.On the other hand,the sufficient descent of error function is guaranteed by constructing a specific conjugate coefficient.2.Under weakening conditions,the weak convergence and strong convergence of the real-valued conjugate neural network algorithm based on generalized Armijo search method are strictly proved.In addition,the effectiveness and rationality of the algorithm are further verified by two numerical experiments.3.For the single hidden layer fully complex neural network,a fully complex conjugate neural network algorithm based on generalized Armijo line search is proposed.By using conjugate gradient method and generalized Armijo search strategy,the algorithm greatly improves the convergence and ensures the sufficient descent direction of the algorithm by constructing a specific conjugate coefficient.4.Under relaxed conditions,the deterministic convergence of the fully complex conjugate neural network algorithm based on the generalized Armijo search method is strictly proved.The rationality of the algorithm is verified by a numerical experiment and the advantages of the algorithm are illustrated by the comparative experiment.