Research And Application Of Weight Initialization Of Convolutional Neural Network

Currently,convolutional neural networks have achieved great success in many fields.In order to further improve the efficiency of convolutional neural networks,researchers have proposed improvements from multiple perspectives,including: reducing network overfitting,optimizing network structure,cross-domain transfer learning,and efficient model weight initialization.At this stage,building an excellent network model usually requires a lot of experiments,and the setting of model parameters often depends on the experience of the researcher,and there is no good theoretical guidance.For the weight initialization problem of network models,this paper summarizes the distribution characteristics of the pre-trained weights of several typical convolutional neural network models based on statistical methods.From these distribution characteristics,the overall weight values after the model training are presented to the characteristics of the 0-direction offset are smaller than the normal distribution,the standard deviation is smaller,the distribution peak is higher,and the phenomenon of long tail appears locally.Based on these distribution characteristics,imagine whether the network model's weight initialization directly adopts this distribution to improve the efficiency of the network? First of all,in this paper,the normality test is performed on the pre-training weights of the model to determine the association between the weight distribution and the normal distribution,and then the maximum likelihood fitting method and the simulation based on Kolmogorov-Smirnov(KS)statistics and likelihood ratio are used.The combined goodness-of-goods test explores the feasibility of using the power law distribution to fit the pre-training weight distribution,and concludes that the pre-training weight has the property of a local power law.Based on the results of the above research,this paper uses the method of adjusting the normal distribution variance to fit the pre-training weights,and proposes a normal weight initialization scheme based on variance adjustment.In order to explore the effectiveness of the scheme,based on the deep residual network model,this paper built a ResNet32 model,and conducted the first round of experiments on the CIFAR-10 data set to verify the effect of the initial model based on variance adjustment.The results prove that the more similar the adjusted initial weight distribution is to the real distribution of pre-trained weights,the higher the efficiency of the model,and the accuracy is improved to a certain extent compared to the commonly used Xavier and Kaiming initialization.In order to further verify the effectiveness of the normalized model based on variance adjustment in practical applications,the fourth chapter selects the classification problem of high spatial resolution(HSR)remote sensing images,and carries out the second on the SIRI-WHU Google image data set.The experimental verification of the round has improved to varying degrees from the experimental effect and model efficiency.