Research On Neural Network Algorithms Based On Gradient Related Methods And Generalized Inverse

Artificial neural network has extensive applications in many fields such as pattern recognition,function approximation,signal prediction,automatic control due to its satisfying intelligence ability.Out of many kinds of neural networks,single hidden layer feedforward networks with the universal approximation capabilities have been investigated more thoroughly.Back propagation(BP)algorithm based on gradient descent method is the most popular and important learning algorithm for training single hidden layer feedforward networks.However,it is obvious that BP algorithm is not effective enough due to its slower convergence speed and time-consuming process.An efficient learning algorithm based on generalized inverse named extreme learning machine(ELM)has gained increasing attentions with its characteristics of extremely fast learning speed and good generalization performance.In order to combine the benefits of the two kinds of algorithms,we propose an efficient algorithm for training neural networks based on gradient related methods and generalized inverse,and extend it to fractional order and complex domain.The main work is threefold:1.A conjugate gradient and generalized inverse-based neural network learning algorithm is proposed.It uses the conjugate gradient method to iteratively update the weights between input and hidden layers to accelerate the convergence speed,while generalized inverse formula is used to compute the weights between hidden and output layers to save the training time.The experimental results based on a few classification problems show that the neural networks with our algorithm have good generalization.2.A conjugate fractional-order gradient and generalized inverse-based neural network learning algorithm is proposed.We employ the definitions of fractional-order derivatives into the algorithm to accelerate the convergence speed further.The experimental results based on a few classification problems show that the conjugate fractional-order gradient and generalized inverse-based neural network algorithm improves the classification performance compared to the integer-order one.3.A gradient and generalized inverse-based complex-valued neural network algorithm is proposed.We extend the algorithm from the real domain to the complex domain.The experimental results based on complex-valued signals prediction and equalization problems show that the gradient and generalized inverse-based complex-valued neural network algorithm has better performance in solving complex-valued problems.