Research On Deep Belief Networks Optimization Technology

As a new deep neural network learning algorithm,deep learning constructs an artificial neural network model with multiple hidden layers by simulating the working methods of the human brain.It extract features from high-dimensional input data layer by layer and then an abstract high-level representation is formed,showing a strong learning ability.Deep learning aims at constructing deep-level structured models.The number of hidden layers is up to hundreds of thousands.This type of multi-hidden neural network structure is difficult to work with traditional training algorithms,not only because of the large amount of data required for training samples.The training process is slow,and the cost function easily converges to a local minimum,making the model ineffective.Until 2006,Hinton et al.proposed a deep belief network,which solved the difficulty of deep neural network training through stratified training and achieved great success.Deep belief network has excellent feature extraction capabilities.It is composed of a plurality of completed pre-trained Restricted Boltzmann machines stacked in sequence.It exhibits excellent performance in prediction and classification and is widely used in various fields.Although the deep belief networks has been successfully applied in many fields,it is still in the development stage and there is still a lot of work to be done.At present,the deep belief networks lacks scientific and effective methods to select the number of hidden layers and number of neurons in the network.Practical applications are mostly based on the experience of scholars to select.If the selected value is too small,the model structure is too simple to fit a complex data structure,making the model less accurate.If the selected value is too large and the model is too complex,it will result in difficulties in model training,low computational efficiency and overfitting,which is not conducive to extended application of the deep belief networks.In response to the above issues,the following studies have been conducted in this paper:(1)According to the idea of greedy algorithm,a method for constructing a deepbelief network model dynamically is proposed: In the process of constructing the deep belief network model layer by layer from the bottom layer,the number of neurons in each layer is dynamically adjusted according to the error classification rate of the verification set until the model meets the accuracy requirement.Finally,the number of neurons in the hidden layers is fine-tuned.(2)For the construction of the deep belief network model based on reconstruction error,the number of hidden layers of the model is often too large;this paper improves it and selects the appropriate number of network hidden layers according to the weighted error;after the network hidden layer number is determined,then according to the increasing Method or dichotomy to determine the number of suitable hidden layer neurons in order to make the model structure better.Through the above research,the deep belief network model can select the appropriate hidden layer number and number of neurons according to this method,optimize the model structure,and improve the accuracy of model classification or prediction.