The Approximation Capability Of Deep Network Based On Restricted Boltzman Machine

In mathematics,the learning ability of neural networks can be regarded as a function approximation problem.From this point of view,we can regard the deep neural network as a function approximator.In practical applications,many deep network models are probabilistic generation models,from the point of view of probability,the learning ability of neural network can also be regarded as a probabilistic distribution approximation problem.In other words the output of the deep neural network should approximate the probability distribution of a random variable.This paper focuses on the theoretical study of neural networks.We analyze the approximation of Restricted Boltzmann Machine(RBM)and Deep Boltzman Network(DBM).Previous theoretical works mainly concern how the representation power is improved with the increasing of the units and the layers.Actually,it is well known that usually more layers and more units are not good options since large parameters will lead to over-fitting.For disussing this problem,this paper give two explanations.One is that when the approximation ability of RBM is optimal,how increasing the number of hidden units has a negative impact on the approximation ability of RBM.The other is that when the approximation ability of DBM is optimal,how increasing the number of network layers may have a negative impact on the approximation ability of DBM.This paper is divided into three chapters: Chapter 1 introduces the research background and significance of this paper,and the research status of neural network approximation analysis.Chapter 2 introduces the concepts and network structure of RBM,DBN and DBM.We also introduce RBM,DBN and DBM as general approximators in detail.Chapter 3 presents the main conclusions and proofs of this paper.