Deep Neural Network Optimization Algorithms Based On End-to-end Method

In recent years,deep learning technologies attract more and more attention from academia and industry.This thesis focuses on optimization for two kinds of deep neural networks,convolution neural network and graph convolution network.They represent different approaches to process tasks under Euclidean distance and non-Euclidean distance.The convolution neural networks have developed their structures from the initial 5layers to hundreds of layers.But most of these networks focus on tasks such as image classification and recognition.Hundreds of different network structures have been extended to solve dozens of sub-field tasks.It costs academia and industry a lot to solve the above problem to improve performance on precision and speed.However,it needs a new neural network when a new problem is proposed.Therefore,this thesis focuses on the network architecture search algorithm.Network architecture search,a new network is generated through another network without human intervention.This end-to-end network optimization algorithm will free people from constructing hand-crafted networks and generate better and faster neural networks.Based on the research of cell performance,this thesis provides a robust cell module for neural network structure search to improve the performance on mainstream datasets.Meanwhile,considering that some networks are easy to fall into the bad performance range at the beginning of training,this thesis adopts the partial channel connection approach,which can not only avoid the early under fitting,but also accelerate the network search speed.Next,this thesis makes some works in the field of graph convolution network,which performs similar convolution operations after the mapping of the inverse Fourier transform.However,we notice that one of the inputs of the graph convolution network is a predefined adjacency matrix,which cannot represent the real relationship between nodes.Therefore,this thesis proposes an algorithm to optimize the matrix automatically,which makes the network define its own adjacency matrix autonomously.There is another advantage of our work,that is our proposed method can handle the definition of a matrix with a more larger input data.The method in this thesis achieves state-ofthe-art performance on Cora and Pubmed datasets,respectively,with the accuracy of84.6% and 81.6%.