Research On Interpretation Technology In Deep Neural Network

Deep neural network(DNN)has succeeded in many research fields,such as audio recognition,computer vision and so on.However,there is a lack of generation theory for DNN.The research of generation for DNN includes representation power,optimization and generation analysis,and the generation analysis is the most significant part of DNN.Given that the DNN is over-parameterizing and different local minimum has different generation power,the classical learning theory fails to interpret the generation in DNN.To solve the generation in DNN,this paper will analyze the generation problem in facial recognition models with the help of PAC-Bayes boundary theory.However,one practical problem of PAC-Bayes boundary theory is the estimation of the prior and posterior distribution of uncertain models.Thus,we attempt to use a parameter-perturbed scheme instead of the posterior distribution of model parameters,and further derive a meaningful generation boundary.Then,a novel parameter-perturbed scheme is introduced in this paper.The innovations and contributions of our work are shown as follows.(1)We improve the PAC-Bayes boundary of facial recognition models.Facial recognition models are mainly based on convolutional neural network(CNN).We will try to derive the spectral norm boundary for parameter-perturbed matrix considering the input and output channels,kernel size and feature map dimension of corresponding models.Further,we simulate the posterior distribution of concept space,and calculate KL divergence as well as PAC-Bayes boundary.(2)We connect the generation power of models and the sharpness of DNN solutions using the PAC-Bayes boundary theory.We prove the relevance among model generation,sharpness of parameter space,norm constraint,input domain and the maximum boundary distance in classification.(3)We propose a novel optimizing-perturbation scheme.The scheme constrains the generation error for the posterior distribution of parameters in models.Since the adding noise in training stage reveals the posterior distribution of current parameters,we update every parameter with the gradient of loss functions that base on the posterior distribution in every iterative step of stochastic gradient descent.Further,the variance of noise-perturbing distribution is subject to the optimal PAC-Bayes boundary.This paper establishes a strict analysis method for generation boundary in facial recognition models utillizing PAC-Bayes boundary theory.Different from the simple metric for the amount of parameters,PAC-Bayes generation boundary remarkably interpret the generation of DNN by constraining the norms of parameters.The simulation results are compared with other generation boundary theories,and we find the improved PAC-Bayes boundary constraints reduces with 1-2 order of magnitudes,and thus compactly constrains the generation boundary of models.The proposed parameter-perturbing scheme is also verified utilizing the facial recognition models,and the results show that a better generation performance is realized.