Research On Conjugate Gradient Learning Algorithms For Complex-Valued Neural Networks

Complex-valued neural networks are an important branch of artificial neural networks,and have been widely used in engineering,biomedicine,and physics.According to the choice of activation functions,complex-valued neural networks can be classified into two cateqories:split complex-valued neural networks and fully complex neural networks.By using bounded but non-analytic split complex-valued activation functions,split complex-valued neural networks can overcome the singularity problem of fully complex neural networks,but cannot effectively capture the coupling information between channels.Therefore,in this thesis we only concern fully complex neural networks.Complex gradient learning algorithm is an important learning algorithm for complex-valued neural networks.However,it still suffers form problems such as slow convergence and easily trapping into the local minima.In order to further improve the performance of the complex gradient learning algorithm,in this thesis,a complex Fletcher-Reeves conjugate gradient algorithm with Barzilai-Borwein stepsize and two complex spectral conjugate gradient algorithms are proposed to train fully complex neural networks Based on the Wirtinger gradient operator,the global convergence results of the proposed algorithms are also established.The main contributions of this thesis can be listed as follows:(1)The complex Fletcher-Reeves conjugate gradient algorithm is improved by using the complex Barzilai-Borwein stepsize and the complex-valued Fletcher-Reeves conjugate coefficient.Combining with the complex-valued Wolfe-type line search and the convergence criterion of the conjugate gradient algorithm,we theoretically prove the global convergence of the complex FR conjugate gradient method with Barzilai-Borwein stepsize.(2)In order to further improve the performance,we proposed two complex spectral conjugate gradient algorithms by considering the interaction between the parameter coefficients in the search direction.The convergence analysis of one of the two proposed algorithms is provided based on mild assumptions.(3)The performance of the proposed algorithms is evaluated by simulation examples.The simulation results confirm the advantages of the proposed algorithms,and also show the difference in convergence behavior of the three algorithms.