The Research On Enhancements Of Deep Convolutional Neural Networks

In recent years,the deep convolutional neural network has developed into a basic common technology,which plays a key role in promoting the development of upper computer vision tasks.Therefore,the research on enhancements of deep convolution neural networks can further enable the upper computer vision tasks,which has the important research significance and the wide application value.The key to the enhancements of deep convolution neural networks lies in three aspects: the network architecture,the training method,the quantity and quality of data.The network architecture determines the learning capacity of the network model,the training method ensures the learning and the convergence of the network model,and the quantity and the quality of data promote the network model to learn the internal pattern of the data.Therefore,this dissertation focuses on the deep convolution neural network,proposes the following four works based on the network architecture,the training method,and the quantity and quality of data.(1)Parameter distribution optimization for CNNs.The parameters of the deep convolution neural network directly affect the performance of the network.However,the relationship between the distribution of parameters of the network and its performance is still an unknown problem.How to measure and use the correlation between them to enhance the network's performance is a valuable research problem.To tackle this problem,this dissertation proposes a parameter distribution optimization method,which proposes an energy function to represent the parameter distribution of the deep convolution neural network,and explores the relationship between the parameter distribution and its performance under strict size constraints(the networks contain the same number of learnable parameters).A large number of experiments show that there is a positive proportional relationship between the energy of convolutional neural network parameter distribution and its performance,that is,the higher the energy of convolutional neural network parameter distribution,the better the corresponding network performance.Therefore,a simple and effective guide for designing networks is proposed,which uses the balanced parameter distribution to design the convolutional neural network.Experimental results show that the deep convolution neural networks optimized by parameter distribution equalization obtain performance improvements on image classification datasets.(2)Training CNNs based on curriculum learning.The training of a deep convolution neural network depends on the mini-batch stochastic gradient descent algorithm,which is different from the gradual learning of human beings.Taking advantage of curriculum learning to guide the training of deep convolution neural networks is still an open problem.In this dissertation,starting from the artificial design of the curriculum,we propose a cumulative data complexity measurement and a progressive learning scheme.Experimental results show the effectiveness of our proposed data complexity measurements and the progressive learning scheme.However,due to the super parameter adjustments,the artificial design curriculum can not be widely applied.This dissertation further proposes an autonomous curriculum design method based on reinforcement learning.Based on the data of the current batch and the state of the network,this dissertation uses reinforcement learning technology to learn a curriculum policy,which can automatically select the appropriate samples to train the neural network.Experiments show that the curriculum policy can effectively guide the training of the neural network.(3)Learning to recognize noisy data via GNN for training robust CNNs.Massive data promotes the development of deep convolution neural networks.However,the various noises contained in these data hinder the performance improvements of CNNs.How to identify noise data and reduce its impact,and then enhance the network performance is an urgent problem.To solve this problem,this dissertation learns to recognize noisy data via GNN for training robust CNNs.This dissertation first uses a meta-learning method to train a backbone network on the noisy dataset and constructs a graph neural network that includes noisy data and clean data from a small clean validation dataset.We find that among the adjacency relationship between data learned by the graph neural network,the distributions of adjacency relationship of data with correct labels are balanced,while the distributions of adjacency relationship of data with noise labels are unbalanced.Based on this discovery,this dissertation uses the adjacency relationship to recognize noise data.Meanwhile,an additional weighting method is proposed to improve the estimation of noise data weight of the meta-learning method.Experimental results show that the proposed method can effectively identify noise data,reduce the impact of noise data on the network,and train robust convolutional neural networks.(4)Learning to restore compressed data for training better CNNs.Image compression is a necessary step in visual information processing.However,the negative impact of image compression loss on deep convolution neural networks has been ignored.How to alleviate this negative impact and enhance networks' performance is a valuable research problem.To solve this problem,this dissertation proposes a compressed image restoration method for training better convolutional neural networks.The proposed method is a joint enhancement framework including the restoration module and the task module,which aims to restore and enhance the compressed data under the requirements of the task.The restoration module is a pixel-level mapping network to learn to recover the information loss during the process of compression.The task module is a network model for different tasks,which guides the restoration module to restore the compressed data.Experimental results demonstrate that the proposed method can effectively restore and enhance the compressed image,improve the quality of the compressed image,and then improve the performance of the convolutional neural network on the corresponding dataset.In addition,because of the low coupling characteristics of the two modules in the joint enhancement framework,the method can effectively deal with different application scenarios of using compressed images.