The Research On Internal Representation Extraction And Structure Optimization Of Deep Neural Network

Today,when the field of deep learning is booming,people often construct complex deep neural networks to train data and get better models than traditional machine learning algorithms.But before the training begins,there will always be a neural network problem of how to construct a suitable structure,which is crucial for the training process and model performance.The structural optimization problem of neural networks has always been a difficult point in the field of deep learning.In order to select a neural network with suitable structure,people need to know the superiority and inferiority of different structural neural networks to the objective function representation,and then make choices,which requires extracting the internal representation of neural network to the objective function that has different structures.The internal representation of the neural network refers to the representation of the objective function in the process of fitting the neural network to the external data.Its external description is about the functional mapping relationship between the input and output signals.This thesis aims to extract the internal representation of the deep neural network and show the function mapping of its external representation.At the same time,the internal representation of the neural network is used to further explore the problem of optimization of the neural network structure.The main work is as follows:(1)The internal representation of the neural network is extracted.In order to facilitate the calculation,ReLU is used as the activation function.Due to the characteristics of the self-segmentation of the ReLU function,the function mapping relationship represented by the neural network is in the form of a piecewise function.In the process of solving this piecewise function,it is necessary to know each of the mapping relationships and the corresponding input signal value range,so we need classify each neuron in the network according to the activation state(when the number of neurons is N,there are 2^N classification cases),in each case,respectively obtain a functional expression of the final output result with respect to the input signal,and at the same time obtain the constraint inequality condition that the input signal satisfies through different state neurons.Under the multi-constraint condition,the linear programming idea is used to obtain the range of values of each unknown variable in the input signal.Finally,discard the contradictory situation and summarize all the existing classifications to get the entire piecewise function mapping relationship.(2)Experimental examples of neural network structure optimization.Select multiple neural network instances with different height and width structures,so that they can be trained on common functions and actual problem data sets,and use the internal neural network representation method to obtain the internal representation of the objective function of different structural neural network instances.Compare the advantages and disadvantages of the internal representations,and then select the neural network examples of the appropriate structure.In this thesis,from the perspective of extracting the internal representation of different structural neural networks,the internal representation of the objective function of deep neural network under different width and height structures is obtained,and then the neural network with appropriate structure is selected to realize the neural network structure optimization.