Research On Optimal Design Of Deep Belief Network And Its Applications

Deep learning(DL)applies multi-layer network structure to extract hierarchical features from data,enabling computers to build complex concepts from simple concepts.Different from traditional machine learning,DL can extract effective feature representation from data and obtain high processing performance.Therefore,it has been widely used in many fields.Deep Belief Network(DBN)is inspired by the cognitive reasoning processes of human brain,and is one of the most successful deep models for DL applications so far.It consists of multiple Restricted Boltzmann Machine(RBM)stacked sequentially.The learning process of DBN mainly includes two phases:unsupervised pre-training and supervised fine-tuning.This phased training strategy has made it a great success in learning deep structure,and has gradually become a research focus in academia and industry.However,there are still many difficult problems in the learning algorithm and application of DBN,mainly including:1)In the data input stage,it is difficult to deal with the autocorrelation between the series data,resulting in the network predicted value lags behind the true value,and the accuracy of the model is significantly reduced.2)In the data input stage,there are many characteristic variables in the input data of research object,and the relationship between the variables is complex and mutual influence,resulting in too many neurons in the input layer,which leads to the increase of network structure complexity and the low accuracy of the model.3)In the process of unsupervised layer-by-layer pre-training for DBN,layer-by-layer compression of features causes the loss of information in high hidden layers,which leads to the degradation of model accuracy.4)In the process of unsupervised training,the momentum and learning rate in the super parameters are difficult to determine,slow convergence speed due to random weight initialization,and the coupling effect between hidden layer neurons leads to the over-fitting problem of training.5)When processing continuous data in unsupervised training stage,the continuous function cannot be approximated well and the network structure is difficult to determine,leading to the low model accuracy.6)In the practical application of DBN in air quality prediction,it is difficult to achieve good precision due to the complex nonlinear,random and non-stationary characteristics of PM_2.5in atmospheric environment.Since the research of DBN data input stage and unsupervised learning stage is the focus of DBN,and it is also the key to affect the learning performance of DBN.Therefore,it is of great practical significance to carry out the research on its optimization design and application.The work and innovation of this paper are as follows:1)Design of deep belief network model based on modified ensemble empirical mode decomposition(MEEMD-DBN-SA)Aiming at the problem that DBN can't deal with the autocorrelation between sequence data well,which leads to the problem that the predicted value lags behind the real value and the prediction accuracy is low,a DBN model based on modified ensemble empirical mode decomposition(MEEMD)is designed.Firstly,a MEEMD algorithm is designed to decompose the input sample data signal and obtain multiple combinations of intrinsic mode functions(IMFs)to eliminate the autocorrelation between the data effectively;Secondly,a DBN model based on simulated annealing(SA)algorithm is established.By optimizing the model structure,the IMF component model is established to make effective prediction.Then,the predicted values of DBN are reconstructed and added to obtain the final predicted results of the model.Finally,the designed model is verified by experiments on the prediction of atmospheric CO₂concentration and total phosphorus in wastewater treatment process.The experimental results illustrate that this proposed model can alleviate the phenomenon that the predicted value lags behind the real value and improve the prediction accuracy.2)Design of deep belief network model based on information correlation strategy and minimum vertex coverage(ICS-MVC-DBN)Aiming at the research object of DBN,there are many characteristic variables in the input data,and the complex relationship between the variables affects each other,leading to complex network structure and low prediction accuracy.A prediction model of DBN based on information correlation strategy(ICS)and minimum vertex coverage(MVC)is designed.Firstly,the maximum information coefficient(MIC)is used to evaluate the correlation between the characteristic variables of the input data and the network output variables,and the characteristic variables with small correlation are deleted.Secondly,the correlation between the remaining characteristic variables are evaluated based on MIC,and the redundant variables are selected,and other variables are saved as the variables to be selected.Then,the redundant variables are selected based on MVC algorithm,and the most representative variables are selected and put into the variables to be selected to complete the elimination of redundant characteristic variables.Then,all the selected variables are input into DBN to complete the construction of the model;Finally,the designed model is verified by experiments on the prediction of total phosphorus in the effluent of sewage treatment process and the prediction of concrete compressive strength.The experimental results show that the model can reduce the complexity of network structure and improve the prediction accuracy.3)Design of deep belief network model based on layer-wise data augmentation(LWDA-DBN)Aiming at the process of layer by layer pre-training of DBN,the layer-by-layer compression of features results in the loss of high hidden layer information,leading to the degradation of prediction accuracy.A prediction model of deep belief network based on layer-wise data augmentation(LWDA)is designed.Firstly,a linear interpolation data augmentation method is designed to expand the input sample data of the visible layer to generate virtual data,and the original input data and virtual data are used as the input data of the visible layer,so as to realize the sample number augmentation of the input of the visible layer.Secondly,linear interpolation data augmentation method is used to expand the visible layer output data again to generate virtual data.At the same time,the output data and virtual data are used to pre-train the first hidden layer,so as to obtain the corresponding feature data in the first hidden layer.Then,the acquired feature data is regarded as the input feature data of the second hidden layer,the data augmentation is continued to generate more rich and diverse feature samples to realize the compensation of information loss of the hidden layer,and the second hidden layer is pre-trained until the pre-training of all hidden layers is completed.Finally,the designed model is utilized to predict the atmospheric CO₂concentration and wind speed.The experimental results demonstrate that the model can improve the prediction accuracy by reducing information loss.4)Design of deep belief network model based on variable hyperparameters and dropout algorithm(VSP-DR-DBN)Aiming at the unsupervised pre-training process of DBN,it is difficult to determine momentum and learning rate in hyperparameters,slow convergence rate due to random weight initialization,and the coupling effect between hidden layer neurons leads to over fitting of training.A DBN model based on variable hyperparameters and dropout algorithm is designed.Firstly,a hyperparameter adjustment strategy based on variable momentum and variable learning rate is designed to automatically adjust the momentum and learning rate in the unsupervised pre-training stage of DBN.Secondly,an improved weight initialization method is used to initialize the weight parameters of DBN to avoid the network falling into local optimum.Then,the coupling effect of hidden layer neurons is reduced based on dropout algorithm,and the contrast divergence algorithm is used for training.Finally,the proposed model is verified by experiments on Lorentz chaotic sequence,CATS artificial time series data set,and wind speed forecasting.The experimental results show that the model can automatically adjust the momentum and learning rate in the hyperparameters,and increase the convergence speed and prediction accuracy.5)Design of enhanced deep belief network model based on adaptive mutation particle swarm optimization algorithm(EDBN-AMPSO)Aiming at the problems that deep belief networks cannot handle continuous data well and the structure is difficult to determine.An enhanced deep belief network(EDBN)model based on adaptive mutation particle swarm optimization algorithm(AMPSO)is designed.Firstly,the DBN is improved by Gaussian noise transformation to enhance its ability of processing continuous data.Secondly,the contrastive divergence algorithm is used to train the network.Then,an AMPSO algorithm is designed to optimize the network structure and obtain the optimal network structure.Finally,the designed model is verified by experiments on Lorentz chaotic sequence prediction,Prediction of ammonia nitrogen in effluent of wastewater treatment process,and nonlinear dynamic system identification data sets.The experimental results show that the model can effectively deal with continuous data and automatically determine the optimal network structure,and improve the prediction accuracy.6)Application of deep belief network on air quality predictionPM_2.5is an important index to evaluate air quality.Accurate prediction of PM_2.5concentration is not only helpful for relevant departments to provide early warning of air pollution level for residents'travel and activities,so as to protect people's life and health,but also has very important practical significance for the governance and protection of the atmospheric environment.Aiming at the problem that PM_2.5in the atmospheric environment is complex,nonlinear,non-stationary and difficult to achieve accurate prediction,this paper comprehensively compares and analyzes the models designed in the previous paper,and selects ICS-MVC-DBN model and VSP-DR-DBN model with good applicability to apply to PM_2.5prediction.In order to verify the effectiveness of the designed model objectively and truly,the PM_2.5concentration data of Hangzhou city is taken as the research object for prediction,the experimental results show that the designed model can achieve accurate prediction of PM_2.5.