Convergence Of Learning Algorithms For Augmented Fully-complex Neural Network

A complex-valued neural network(CVNNs)is a very attractive field at the end of the 1980 s and applicable to optoelectronics,imaging,remote sensing,quantum neural devices and systems,and artificial neural information processing.Based on the nature of signals(i.e.,the characteristics of input signals,the output signals and the weight parameters),which are real-valued or complex-valued,the neural networks are classified into real-valued neural networks or complex-valued neural networks.Complex-valued neural networks are divided into two classes,split complex-valued neural networks and fully complex-valued neural networks,in accordance with the difference of their activation functions.The activation functions of split complex-valued neural networks are split complex-valued functions.Although the boundedness of neurons’ outputs can be guaranteed,the activation functions can’t satisfy the Cauchy-Riemann condition,so they don’t have complex-valued derivatives.The activation functions of fully complex-valued neural networks are analytical so that their complex-valued derivatives exist.This paper mainly studies fully complex-valued neural networks.This paper is focused on two aspects.The first aspect is to present an augmented algorithm for fully complex-valued neural network based on Wirtinger calculus,which simplifies the derivation of the algorithm and eliminates the Schwarz’s symmetry restriction on the activation functions.An unified meanvaluetheorem is established for general functions of complex variables,covering the analytic functions,non-analytic functions and real-valued functions.Based on this theorem,several convergence results of the augmented algorithm isobtained under mild conditions.Simulations are provided to support the analysis.The second aspect is to analyze and research the problem,which is about the convergence of with inner-product penalty and with augmented weight-decay penalty of the batch gradient algorithm in two layer CVNNs.Theoretical proof and derivation are provided at the same time.The content of this paper can be divided into four parts.Firstly,the related knowledge of CVNNs and the research status at home and abroad are introduced.Secondly,The complex-valued neural network theory is expounded,which contains theory,complex Wirtinger calculus theory,complex differential,complex Jacobian,application of Wirtinger operator theory.Meanwhile two optimization algorithm of neural network are introduced.Thirdly,an augmented algorithm for fully complex-valued neuralnetwork based on Wirtinger calculus is presented.An unified mean value theorem is established for general functions of complex variables.Some convergence results of the augmented algorithm are obtained under mild conditions.Simulations are provided to support the analysis.Lastly,the convergence of thealgorithm withpenalty in CVNNs is analyzed.