| Deep belief network(DBN)is one of the main models of deep learning,which has been widely used in machine learning fields such as speech recognition,handwriting recognition,face recognition and so on.As an effective feature extraction method,DBN can extract the intrinsic abstract features of data,which are more discriminative than the original features.In this paper,the automatic adjustment method of DBN structure is studied.The main research contents are as follows:(1)Since the training state of DBN is influenced by the network structure,it can be adjusted according to the training state.Therefore,a DBN model(DDBN)based on dynamic growing and pruning algorithm is proposed.The training state of DBN is measured by the weight and activation probability of hidden neurons and the energy of the network.DDBN first uses Weight Distance(WD)to calculate the weight change magnitude of each neuron,and a monotone decreasing concave curve is proposed as a constraint condition to find the unstable neurons with obvious variation in weight.Subsequently,the hidden neurons are added around them to improve the learning ability of network.Furthermore,DDBN calculates the standard deviation of hidden neurons activation probability to measure the dispersion degree of the extracted features,and then finds out the redundant neurons with low discriminative ability and removes them to obtain a compact structure.Finally,DDBN uses the energy function to evaluate the stability of the network.DDBN automatically constructs the network structure according to the training state of DBN,which effectively solves the problem of DBN structure determination.The results of image classification tests on MNIST and USPS datasets show that DDBN has better classification accuracy than existing DBN structure adjustments methods.(2)In order to extract more discriminative features,it is necessary to enhance the discriminating ability of neurons.Since the discriminative ability of neurons is affected by the network structure,this paper proposes a DBN model(FDBN)based on Fisher criterion,which enhances the discriminative ability of neurons by optimizing the network structure.Firstly,FDBN calculates the inter-class dispersion and intra-class dispersion of all training samples on each hidden layer neuron,and the Fisher value of the neuron is obtained by the ratio of the two terms.The larger the Fisher value is,the feature extracted by the neuron is more discriminative.Then,the discriminant performance of neurons is evaluated according to the changes of Fisher value and growth rate of neurons in iterative training,thereby adding neurons or pruning redundant neurons.In addition,FDBN uses the greedy algorithm to adjust the number of hidden layers according to the classification effect of the current network.FDBN uses label information in the process of evaluating neuron discriminative performance,and adjusts the network structure to extract more discriminative features.Finally,handwritten digit recognition tests are performed on MNIST and USPS datasets.The results show that FDBN always achieves the best recognition accuracy.(3)Aiming at the problem that the number of hidden layers of DBN is difficult to determine,a DBN model(DTDBN)based on depth trend is proposed.Firstly,according to the central limit theorem and information entropy theory,the change trend of the weight of DBN with the increase of depth is analyzed.The weight distribution will gradually tend to normal distribution,and the closer the weight is to the normal distribution,the better the feature extraction effect of the network.Furthermore,DTDBN uses normality test to calculate the obedience rate of the normal distribution of the weight,and then to determine whether a new hidden layer needs to be added.According to the depth trend of the network,DTDBN effectively adjusts the network depth by using the normal distribution obey rate of the weight,and improves the expressive ability of the network.Finally,a DTDBN face recognition model is constructed,and the experimental analysis on ORL database proves that DTDBN has better classification accuracy. |
