The Study Of Fuzzy Deep-learning Neural Networks Algorithms

The discipline of deep learning networks has been gradually open up with theproposing of the basis of the stack of the Restricted Boltzmann Machine (RBM) ofdeep belief networks learning algorithm since2006. It is a new field in the artificialintelligence. The main content of the field is the multilayer neural networkmodeling and the algorithm learning problems. The applications of the field havebeen very broad so far, such as the phone’s touch screen, handwriting input methodand so on, and it has broad prospects for development.This article focused on the study of the algorithms of the fuzzy deep learningneural networks, we especially focused on the basis of the deep learning networkswhich is the RBM.First, we have introduced the research background, the development situationsof the neural networks, fuzzy neural networks and deep learning algorithms and soon. We have summarized that the most important deep learning RBM learningalgorithm is based on the Markov chain Monte Carlo (MCMC) learning algorithms,however, the purpose of these algorithms is for sampling, so it is very necessary forus to propose a more efficient learning algorithm from the deep learning itself.Secondly, we have systematically introduced the theoretical knowledge of theRBM and the deep belief networks. On the RBM part, we have introduced themodel structure of the RBM and the theoretical analysis of the model parameters.And on the deep belief networks part, we have introduced the basic components ofthe network and the problems of the two layers of the RBM processing method.Third, we have focused on the comparative analysis of the main study learningalgorithms of the RBM in deep belief networks, including the ContrastiveDivergence algorithms and Stochastic Maximum Likelihood (SML) algorithms;tempering MCMC algorithms including the Simulated Tempering algorithm,Tempered Transition algorithm and Parallel Tempering (PT) algorithm and so on;mode-hopping MCMC algorithm and so on. In addition, we also have studied theMarkov chains and the Gibbs sampling algorithm; the assessment algorithms of theRBM, including the reconstruction error algorithm and the Annealed ImportanceSampling algorithm.Finally, we have put forward the concept of the “mode set” based onmode-hopping MCMC algorithm and purpose a mode-hopping MCMC algorithmbased on the “mode set”. We have proved that the algorithm based on the mode setis more efficient than the algorithm without the mode set, the algorithm would jumpquickly among the mode sets with the increasing of the dimension of the hidden nodes, we have learned that the improved algorithm can store more set informationin spite of the decreasing of the mode set radius for the jumping speed of thealgorithm between different neighbor mode set radius. Then, we have compared theimproved algorithm with the Simulated Tempering algorithm and the ParallelTempering algorithm, and we have concluded that the improved algorithm is moreefficient. At last, we have used Annealed Importance Sampling algorithm to assessthe above three algorithms, and we have concluded that the likelihood of ourmode-hopping MCMC algorithm based on the mode set is the largest. Therefore, theimproved algorithm is much more efficient in learning, so the designing direction ofour improved algorithm is correct.