Deep Learning For Generative Model And Numerical PDEs

This dissertation focuses on two applications by using deep learning:numerically solving PDEs and implicit generative learning.Firstly,we give a theoretical analysis for deep neural network solving PDEs.In re-cent years,physical informed neural networks(PINNs)have been shown to be powerful tool for solving PDEs empirically.However,numerical analysis of PINNs is still miss-ing.In this dissertation,we prove the convergence rate to PINNs for the second order elliptic equations with Dirichlet boundary condition,by establishing the upper bounds on the number of training samples,depth and width of the deep neural networks to achieve desired accuracy.The error of PINNs is decomposed into approximation error and statistical error,where the approximation error is given in C~2norm with Re LU~3networks and the statistical error is estimated by Rademacher complexity.We derive the bound on the Rademacher complexity of the non-Lipschitz composition of gradient norm with Re LU~3network,which is of immense independent interest.Generative models are used to learn unknown distribution from given datasets.Ex-plicit approaches often result in curse of dimensionality and high computational cost,while implicit ones will not suffer from these problems.The proposed approach is moti-vated by the problem of finding an optimal transport map from a reference distribution to a target distribution characterized by the Monge-Ampere equation.Interpreting the infinitesimal linearization of the Monge-Ampere equation from the perspective of gradi-ent flows in measure spaces leads to a stochastic Mc Kean-Vlasov equation.We use the forward Euler method to solve this equation.The resulting forward Euler map pushes forward a reference distribution to the target.This map is the composition of a sequence of simple residual maps,which are computationally stable and easy to train.The key task in training is the estimation of the density ratios or differences that determine the residual maps.We estimate the density ratios based on the kernel method.Numerical experiments support our theoretical results and demonstrate the effectiveness of the proposed method.Then,to better learn the latent representation of complex data,we further propose a bidirectional gradient flows(BGF)approach for deep generative learning.Unidirec-tional gradient flow evolves from reference distribution to target distribution.In BGF,there are two flows evolving in the opposite directions:the generator flow consisting of particles that evolve from the latent distribution to the data distribution;the en-coder flow consisting of particles that evolve from the data distribution to the latent distribution.These flows are governed by common velocity vectors,which are deter-mined by the density ratio between the distributions of the encoder and the generator flows.We estimate the velocity vectors through estimating the density ratios from the particles in both flows.BGF is trained stably via the composition of a sequence of residual maps which are perturbations of the identity map along the estimated velocity vectors.Extensive numerical experiments are conducted to evaluate BGF and demon-strate that BGF performs better than or similarly to the state-of-the-art bidirectional learning methods.In summary,we have employed deep learning methods for PDEs and generative learning,which give new insights for classical machine learning task and numerical computation theorem.