Improvement Of Neural Autoregressive Distribution Estimation And Its Classification Application

The neural autoregressive distribution estimation,NADE solves the problem that the partition function in the high dimensional distribution is computationally complex with restricted Boltzmann machine.The NADE model is proposed by the combination of the RBM model and Bayesian belief network,which can easily calculate the joint density distribution.If you stack multiple NADE to form a deep Neural Autoregressive Density Estimate,you can extract more abstract features.Based on the basic model of NADE,this paper,we study NADE model from the perspective of regularization to the network structure sparsity and the parameters updating idea.The main results are as follows:1.On the basis of the NADE model,the NADE model is improved with the1 L norm sparsity,combined with the advantages of Polyak Averaging idea to stabilize fast convergence in the parameter updating.The sparse NADE model is further optimized,and the UCI dataset is selected to test the probability distribution fitting ability of the improved algorithm.The results showed that the fitting ability was improved.2.Applying the parameter optimization idea PAS to the extended model Sup Doc NADE of NADE,the PAS-Sup Doc NADE model is proposed.Comparison between original model and improved model by image classification experiment,the dataset Label Me classification accuracy is increased from 83.43% to 83.85%,and the dataset UIUC-Sports classification accuracy is increased from 77.29% to 78.12%.3.To study the improvement of the classification performance of the PAS-Sup Doc NADE model with the increase of the number of hidden layers,this paper studies the improvement of the classification accuracy of the data set after adding two layers of hidden layer and three-layer hidden layer.First,the Deep Doc NADE model is used to train the parameters of the PAS-Deep-Sup Doc NADE model,and the results show that the classification accuracy is improved.