Research And Application On Feature Extraction Algorithm Based On Deep Neural Network

At present,deep neural network has become a research hotspot in the field of machine learning,it is essentially the process of extracting feature.The biggest advantage lies in allowing the calculation model consisting of multiple hidden layers to extract the abstarct characteristics of data automatically from layer by layer and learning representative and discriminative features.In addition,the deep neural network can use few parameters to represent more complex functions.For new appplies,it can quickly learn effective feature expression from training data,greatly improving technologies in many other fields such as speech recognition,visual object recognition,object detection and so on.In this paper,the feature extraction algorithm based on deep neural network is analyzed and studied.The main contents of this paper are as follows :1.Aiming at the problem that the deep belief network(DBN)is susceptible to the training parameters during the fine-tune process,this paper proposed a kind of DBN method based on batch normalization(BNDBN).Firstly,this method used unsupervised learning to obtain highlevel representation of raw data.Then through the scale transformation and translation transformation parameters in the BN alogrithm,it processed the output characteristics of each layer by batch normalization.And it fed the post-processing characteristics into the nonlinear transformation activation layer.Finally,it trained and studied the parameters of the affine transformation and the original network in batches by using the mini batch stochastic gradient descent method.The BNDBN method reduced the dependence of the gradient on the parameter sclae.The transformation reconstruction can restore the feature distribution learned by the original network and effectively solve the problem of changing the value distribution of activation function caused by the change of network parameters.To verify the effectiveness of the proposed method,it selected MNIST handwritten database and the USPS handwritten digital identification library for testing.Compared with a variety of typical algoritms,the results show that the proposed method significantly improved the classification accuracy and had stronger feature extraction ability.2.The traditional DBN is trained through layer-by-layer unsupervised learning.However,it is easy to to generate a large amount of redundant information in the training process and the affect the feature extraction ability.In order to make the model more explanatory and discriminative,first of all,a penalty regularization term was introduced in the unsupervised stage of likehood function based on the inspiration of the primate visual cortex analysis.While using the CD algorithm to maximize the objective function,the sparse constraint is used to obtain the sparse distribution of the training set,so that the unlabeled data can be learned to a intuitive feature representation.Secondly,aiming at the problem of invariance existing in sparse regular term,an improved sparse deep belief nerwork is proposed,which uses Laplace distribution to induce the sparse state of hidden layer nodes,and the location parameters in the distribution are used to control the intensity of sparsity.Through validation analysis on the MNIST and Pendigits handwritten data sets,and compared with other existing technology methods,this method consistently achieves the best recognition accuracy and good sparseness even with very few samples per class.3.Aiming at the problem that the gradient disappears easily for the traditional sigmoid function,this paper proposes a nonlinear modified deep belief network(MBDBN).Firstly,we use the Elliott function that satisfies the generalized Logistic differential equation instead of sigmoid activation function.Secondly,in order to ensure that the neurons are in saturation and satisfy the properties of the restricted boltzmann machine during training,the Elliott function is modified and used as an activation function of the model,which has lower computational complexity than using an exponential function and the network is easier to optimize;Further,a face recognition framework based on MEDBN is constructed.The experimental verification on the ORL library proves that this paper has better robustness;In order to test the wide applicability of the algorithm,MEDBN is used for image classification and tested on the MINIST and USPS databases.The recognition accuracy rate is improved,which futher demonstrated that algorithm has good feature extraction capabilities.