Learning And Learning To Discretize Partial Differential Equations

Differential equations,especially partial differential equations(PDEs),play a prominent role in many disciplines.Partial differential equations are commonly derived based on empiri-cal observations,to describe the governing physical laws underlying a given system of interest.Numerical methods and theories can also be developed to simulate or analysis the model.With the recent rapid development of sensors,computational power,and data storage in the last decade,huge quantities of data can now be easily collected,stored and processed.Such vast quantity of data offers new opportunities for data-intensive research,which is well known as a new paradigm as well as experimental,theoretical and(more recently)computational.When we know the expression except for some parameters of the PDE,there are a variety of techniques(namely system identification/PDE inverse problem)to estimate the unknown parameters.However,for many complex systems,such as neurosciences,finance,and life sciences,it is difficult to obtain the model's full expression,and a large number of empirical parameters/formulas are often required.A natural question is whether we can utilize all collected data,to learn the expression and parameters of PDE at the same time?The form of the PDE is usually assumed to be known in classical approaches,thus we can often design reliable forward schemes for the iterative solution(which need to repeatedly solve the forward problem)to the inverse problem.In contrast,when the form of PDE is unknown,the first problem we have to solve is that,how to discretize the forward problem suitably when we do not know the PDE's expression?To this end,we propose to unroll time integration of numerical PDE solvers as DNN(deep neural network)forward propagation or as Markov Decision Process,and then solve system identification problems while jointly learn a proper numerical dis-cretization.Some proof of concept studies^{[100,117,122]}will be introduced.The article shows how we can combine new machine learning approaches,such as deep learning,reinforcement learning(RL),to develop a PDE system identification method.The proposed methods need minor prior knowledge on the underlying mechanism and is able to make full use of spatial-temporal data.We will focus on both transparency and predictive power of the identified model.Instead of one-step-ahead prediction error,we try to minimize multi-step error.This article is divided into three parts:Unfold forward Euler scheme as DNN forward propagation,and approximate differential operations by convolution.It is very important to properly discretize the forward model for inverse problem.Notice that,the commonly used CNN(convolution neural network)have close relationship with nu-merical PDEs.As a proof of concept,we are going to design a feed-forward network,named the PDE-Net^[100,101],that approximates the convection-diffusion equations in the way that:1)unfold forward Euler scheme as DNN forward propagation;2)approximate differential op-erations by convolutions with properly constraint filters;3)approximate convection/diffusion coefficients by piecewise polynomials.The results show that PDE-Net can uncover the coeffi-cients and predict the dynamical behavior for a relatively long time.Reduce model assumptions by utilizing the advantage of DNNs'expressive power for composite functions.In this part of the article,we inherit the framework from PDE-Net in order to maintain the advantage of efficiency of data using.And instead of using piecewise polynomials,a simplified version of EQL/EQL^÷,which is named as Sym Net,was equiped into PDE-Net,and we call this network PDE-Net 2.0^[117].Compared with SINDy,in which requires a large dictionary,the composite representation by Sym Net is more flexible and efficient.Moreover,we adopt pseudo-upwind trick to improve stability of inferencing.PDE-Net 2.0 was tested on Burgers'equation,heat equation and convection diffusion with nonlinear reactive source under the same assumptions on the mechanisms.The experiment results show that PDE-Net 2.0 is able to recover the analytic form of the PDE model with minor prior knowledge on the type of the equation.Furthermore,after training,the network can perform accurate long-term prediction without re-training for new initial conditions.Learning to discretize a given partial differential equation from data.It should be noted that,the use of convolution instead of finite difference and pseudo-upwind trick we mentioned in the first two parts,are proposed to improve stability and expressive power of PDE-Net/PDE-Net 2.0 while maintaining consistency.In this part,we would like to regard numerical PDE solvers as Markov Decision Process(MDP)and use deep RL to learn new solvers.Learning a general type of discretization can also be viewed as a close loop control problem.The proposed method^[122]was tested on 1d conservation laws.Numerical results show that,the RL based solver was able to outperform high order WENO and was well generalized in various case.