Research On Complex Neural Network

At present,most of the deep learning techniques,modules and frameworks are based on real domain.It has been found that complex numbers have incomparable advantages over real numbers,such as richer representational,phase information,and robustness to noise.Although complex networks have such excellent potential,there is a lack of modules to build complex networks,so few people study complex neural networks.In this thesis,the construction method of real neural network is studied,and the composition method of complex neural network is analyzed in detail,so that the neural network can be extended to complex domain.This thesis mainly studies convolutional neural networks and recurrent neural networks,and extends them to the complex domain.In order to study the complex neural network,the convolutional neural network and the recurrent neural network in deep learning are deeply analyzed,including why the recurrent neural network has gradient disappears and explodes problem,and the implementation principle of each network layer in the convolutional neural network.then constructs the complex neural network based on these researches.The main research contents include:(1)Study the implementation mechanism of recurrent neural network based on unitary matrix: the disappearance of gradient or explosion of recursive neural network in backward gradient propagation will make the network unable to continue training.This thesis analyzes the reason why the unitary matrix based recursive neural network can solve the problem of gradient disappearance or explosion,and compares three typical parameterized unitary matrix methods: UERNN,Tunable and FFT,it is found that the space covered by the three methods is the subspace of unitary space,and only Tunable can adjust the size of subspaces by modifying parameters.(2)Research on the construction method of complex deep residual neural network: The advantages of complex number in parameter representation and network depth are studied and analyzed,as well as the construction method of complex residual neural network.In order to implement the data processing on complex deep residual neural network,five key modules of complex residual network are constructed,including complex convolution,complex pooling,complex weight initialization,complex batch normalization and complex activation function.,and the complex residuals network is constructed by using these five modules.In order to verify the advantages of complex neural network,this thesis designs several experiments to verify the performance of complex recurrent neural networks and complex residual networks.Specifically,include:(1)Experiment on recursive neural network based on unitary matrix: Three tasks are designed in this experiment,including copying memory task,denoising task and parentheses task.Three parameterized unitary matrix methods,UERNN,Tunable and FFT,are applied to recurrent neural network respectively,then together with LSTM,GRU and GORU,this experiment test these six networks on these three tasks.Experiments show that the recurrent neural network composed of Tunable performs best on the copying memory task,and the GORU performs best on the denoising task and the parentheses task(2)Experiment on residuals networks in real and complex domains: Two image classification tasks based on CIFAR-10 and CIFAR-100 and music transcription tasks based on Music Net are designed in this experiment.Experiments show that complex network performs poorly in image classification tasks,but its accuracy in music transcription tasks exceeds 3.3% in real residual networks.The performance of complex residual network optimized by nonlocal connection network is also improved by 0.1%.