Study On Highly Accurate Strongly Stable Numerical Solutions And Deep Learning Solutions For Forward Backward Stochastic Differential Equations

In 1990,Pardoux and Peng[63]proved the existence and uniqueness of the solutions for nonlinear backward stochastic differential equations（BSDEs）,which lay the foundation of the theory of BSDEs.In 1991,Peng[66]obtained the famous nonlinear Feynman-Kac formula,which establishes a deep relation between BSDEs and a class of parabolic partial differential equations（PDE）.Since then,the theory of BSDEs has been developing rapidly,and been applied to many fields,e.g.,PDEs[66],financial mathematics[25,69],stochastic control[65],nonlinear expectations[67],etc.The applications of BSDEs deeply rely on their solutions,but the analytical solutions are hardly available for BSDEs,which raise the value of studies on their numerical solutions.In this thesis,we focus on numerical methods for forward backward stochastic equations（FBSDEs）.In recent 30 years,a large collection of literatures have been devoted to study efficient numerical methods for FBSDEs.Nowadays,numerical schemes for FBSDEs include but are not limited to the Euler scheme[32,33,84],θ-schemes[86,89],prediction-correction schemes[28],Runge-Kutta methods[15],linear multistep methods[14,87,90.92],etc.All the above-mentioned schemes have their prototypes in numerical methods for ordinary differential equations（ODE）.The study on numerical methods for ODEs has a long history.In the 20th century,a systematic theory of consistency,stability and convergence has been established for one-step and multistep schemes for ODEs[13,18,19,52].This theory provide a complete theoretical foundation for the design of highly accurate and strongly stable schemes for ODEs.However,for FBSDEs,the corresponding methods and theories are not well developed,especially for high-order and strongly stable numerical methods,where numerous problems remain unsolved.There have been abundant results in the field of numerical ODEs,and thus extending these results from ODEs to FBSDEs has great theoretical and practical significances in improving the numerical theories of FBSDEs.Based on the above research status,in this paper,we will establish new consistency and stability results for general one-step and multistep schemes for FBSDEs by combining the numerical methods and theories of ODEs.Using these results,we will further propose a series of highly accurate and strongly stable numerical schemes for FBSDEs.As an application study,we will combine the proposed schemes with deep learning techniques to solve high-dimensional problems.The Main Contributions and Novelties of this Thesis1.We first establish a relationship between numerical solutions for BSDEs and ODEs.Based on the nonlinear Feynman-Kac formula[66],the BSDE is reformulted into a pair of reference ODEs,which can be directly discretized by standard ODE solvers,leading to the corresponding numerical solvers for BSDEs.This approach greatly facilitates the construction and analysis of numerical schemes for BSDEs.The above research results have been published in Numer.Math.Theor.Meth.Appl.[26].2.We propose a new kind of Runge-Kutta schemes for BSDEs,and establish a method to theoretically analyze the proposed schemes.Specifically,we nontrivially extend the Butcher theory from ODEs to BSDEs,based on which we obtain the order conditions for BSDE’s Runge-Kutta schemes.C ompared which the results in existing literatures,our order conditions enjoy universality,which are applicable to BSDE’s Runge-Kutta schemes with arbitrary stages and arbitrary orders.Based on this,we first obtain BSDE’s Runge-Kutta schemes with order higher than 4.Focus on the proposed Runge-Kutta schemes,we present rigorous theoretical error estimates.The above research results have been included in a paper to be published.3.We adopt new theoretical methods to analyze the consistency and stability of the general linear multistep schemes for BSDEs and FBSDEs,and further obtain theoretical error estimates for the schemes.These new theoretical results are not covered by existing literatures.Based on the above results,we first extend the SSP multistep methods[35-37,54,72,73]from ODEs to BSDEs and FBSDEs,and propose new high-order SSP multistep（SSPM）schemes,which are capable of solving BSDEs and FBSDEs appearing in complex problems.In numerical experiments,the proposed SSPM schemes show high-order convergence and stronger stability than non-SSP schemes.The above research results have been published in J.Sci.Comput.[27]and Numer.Math.Theor.Meth.Appl.[26].4.We first combine high-order time discretization methods for FBSDEs and deeplearning function approximations,based on which we propose deep learning multistep schemes for solving high-dimensional FBSDEs.Compared with competitors in existing literatures,the proposed schemes enjoy high-order time convergence rates,which greatly reduce the number of required time steps.Moreover,in each time step,the computations of our schemes are of high parallelism across the spatial direction.Owing to the above features,our schemes are of high accuracy and efficiency.The above research results have been included in a paper to be published.The Main Results of the ThesisIn this thesis,we focus on the following decoupled Markovian FBSDEs on a complete filtered probability space（Ω,F,F,P）:where t ∈[0,T],b:[0,T]×R^d→ R^d,σ:[0,T]×R^d→R（^d×q）,f:[0,T]×R^d×R^p×R^q×q→R^p,φ:Rq→R^p;T>0 is a deterministic terminal time;W=（Wt）0x≤t≤Tis the qdimensional standard Brownian motion;F=（Ft）0≤t≤T is the natural filtration of generated by W.In（0.17）,the integrals with respect to s are of the Lebesgue type,and the integrals with respect to Ws are of the It? type.On the time interval[0,T],we introduce the following regular partition:where Δtn:=tn+1tn and C0 is a constant independent of πN.In the following,we will introduce the main results in each chapters.For the definitions of the involved notation,readers may refer to the Notation Index（符号说明表）.Chapter 3 Strongly Stable Multistep Methods for BSDEsIn this chapter,we consider a special case of FBSDEs（0.17）,where the following conditions hold:Under the above conditions,by the nonlinear Feynman-Kac formula（2.5）and the formula of integration by parts,the solutions（Y,Z）of（0.17）can be expressed by the following reference ODEs:where Θs and Ft（s,Θs）are given by with Wt,s=Ws-Wt.Based on（0.19）and following the form of ODE’s Runge-Kutta schemes,we propose the explicit Runge-Kutta schemes as follows.Scheme 3.1.1.（Explicit Runge-Kutta Scheme）Given ΘN,for n=N-1,N2,…,0;solve Θn by with tn,i=tn+1-ciΔtn,Here aij,bi and ci are all deterministic scaler coefficients satisfying 0=cm<cm-1<…<c1<c0=1.For n=0,1,…,N-1 and i=1,2,…,m,denote by Rn,i and Rn the local truncation errors of the ith internal stage and the last stage of Scheme 3.1.1,respectively,i.e.,where Θn,i are given by The errors Rn,n=N-1,N-2,… 0 given in（0.20）are also called the local truncation errors of Scheme 3.1.1.By extending the Butcher theory[13]from ODEs to BSDEs,we obtain the Taylor expansions of the local truncation errors Rn,i,i=0,1,…,m shown as the following Theorem 3.3.1,where the notation T denotes a ULN-tree（unlabelled numbered tree）,and the notation |Υ|,α（Υ）,Υ（Υ）,S（Υ）,Ajm（Υ）and Ftn,jΥ denote the order,factorial,symmetry order,elementary coefficients and elementary differentials of T,respectively.The strict definitions and relevant properties of the above notation are given in section 3.2.Theorem 3.3.1.Under Assumptions 3.3.1 and 3.3.2,it holds that for j=0,1,…,m,where and the reminder term in（0.21）holds uniformly for 0≤n≤N-1.The following theorem gives the order conditions of Scheme 3.1.1.Scheme 3.3.2.Under Assumptions 3.3.1 and 3.3.2,if the coefficients of Scheme 3.1.1 satisfy the following rth order conditions then Scheme 3.1.1 is consistent of order r,that is,where Rn are given by（0.20）and are the local truncation errors of Scheme 3.1.1.By Proposition 3.3.3,we give the following sufficient condition for the order condition（0.22）:whereΤr-:={Υ∈Τr:[]0 is not a branch of Υ}.Here the definition of branches are given in Definition 3.2.12.For r≤5,the elements in Τr-are listed as follows Here[]0,[]1,[]2,[[]1]0,… are all ULN-trees given by Definition 3.2.6.Combining Theorem 3.3.1 and 3.3.3,we have the following consistency results.Corollary 3.3.4.Under Assumptions 3.3.1 and 3.3.2,if the coefficients of Scheme 3.1.1 satisfy C（r）in（0.24）,then Scheme 3.1.1 is consistent of order r in sense of（0.23）.We introduce Assumption 3.4.1 for the stability analysis of Scheme 3.1.1 and present the stability result in Theorem 3.4.4.On this basis,combining Theorem 3.3.2 and Corollary 3.3.4,we obtain the following convergence error estimate.Theorem 3.4.6.Suppose that Assumptions 3.3.1,3.3.2 and 3.4.1 hold.Further suppose that the terminal error satisfy E[p2（eN）]≤C|Δt|2r for some constant C independent of πN.If the coefficients of Scheme 3.1.1 satisfy the conditions in（0.22）or the ones in（0.24）,then Scheme 3.1.1 is convergent with order r in sense of where Θn are given by Scheme 3.1.1,and ρ（·）is a seminorm introduced by Assumption 3.4.1.Chapter 3 Strongly Stable Multistep Methods for BSDEsIn the chapter,we still consider the numerical solutions of（0.17）under the condition（0.18）.On the basis of（0.3）and ODE’s linear multistep schemes,we propose the following scheme.Scheme 4.1.1.（Linear multistep schemes for BSDEs）Given {ΘN-i}i=0k-1,for n=N-k,N-k-1,…,0,solve Θn by where α:=[αi]i=1k∈Rk and β:=[βi]i=1k∈Rk are coefficients of the scheme.To obtain the consistency conditions of Scheme 4.1.1,we define the local truncation Rn at tn by where n=0,1.…,N-k.The following proposition gives the consistency conditions of Scheme 4.1.1.Proposition 4.1.1.Let Assumptions 4.1.1.and 4.1.2 hold.If the coefficients of Scheme 4.1.1 satisfy the following k-step rth-order conditions:where δ0j is the Kronecker symbol,i.e.,δ0j=1 for j=0,and δ0j=0 for j≠0,then Scheme 4.1.1 is consistent with order r in sense of for some constant C independent of 7rN,where Rn are given by（0.25）and are the local truncation errors of Scheme 4.1.1.Based on Scheme 4.1.1 and the main idea of SSP multistep scheme[35-37,54,72,73]for ODEs,we propose the following SSP multistep（SSPM）schemes for BSDEs.Definition 4.1.2.（SSPM schemes for BSDEs）A SSPM scheme for BSDEs is the Scheme 4.1.1 with its coefficients a and β satisfying the following k-step SSP condition for some constant ε0 independent of πN where C（α,β）is the SSP coefficient given byDenote by SSPM（k,r）the k-step rth-order SSPM scheme,i.e.,the k-step SSPM scheme satisfying the k-step rth-order conditions in（0.26）.Following the main idea of SSP multistep schemes[35-37,54,72,73]for ODEs,we give the coefficients of the optimal SSPM（k,r）by whereIn Table 4.1,we present some optimal SSPM schemes for BSDEs under uniform time partitions.Under Assumption 3.4.1,we obtain the stability result of BSDE’s SSPM schemes;see Theorem 4.2.2.Further combining Proposition 4.1.1,we obtain the following convergence error estimates.Theorem 4.2.5.Let Assumptions 3.4.1 and 4.1.1 hold.If the terminal values of SSPM（k,r）satisfy where C is a constant independent of πN,then SSPM（k,r）is convergent with order r in sense of where Θn are given by Scheme 4.1.1;ρ（·）is the seminorm introduced by Assumption 3.4.1.Chapter 5 Strongly Stable Multistep Methods for FBSDEsIn this chapter,we focus on the numerical solutions of FBSDEs（0.17）.Compared with the last two chapters,the results in this chapter are more general because they do not rely on the conditions in（0.18）.Scheme 5.1.1.（Linear multistep schemes for FBSDEs）Given {（YN-i,ZN-i）}i=0k-1,for n=N-k,N-k-1,…,0,solve（Yn,Zn）by where fm:=f（tm,Xm,Ym,Zm）,m=0,1,…,N;the vectors α:=[αi]i=1k,β:=[βi]i=0k,α:=[αi]i=1k,β:=[βi]i=1k and γ:=[γi]i=1k,are the coefficients of the scheme.By nonlinear Feynman-Kac formula（2.5）,for n=0,1,…,N,define For n=0,1,…,N-k,define the local truncation errors（Ryn,Rzn）of Scheme 5.1.1 by The rth-order consistency conditions of Scheme 5.1.1 are given by where the matrices Ar+l,k,Br+1,k+1,Ar,k,Br,k,Ar,k and vectors dr+10,dr1,Or are given by（5.24）-（5.27）.The following theorem gives the consistency result of Scheme 5.1.1.Theorem 5.1.2.Under Assumptions 5.1.1,5.1.2 and 5.1.3,if the coefficients of Scheme 5.1.1 satisfy（0.28）,then Scheme 5.1.1 is consistent with order r,i.e.,where（Ryn,Rzn）are given in（0.27）.We introduce the following stability conditions:where ε0>0 is a constant independent of πN.Under the above stability conditions,we give the stability result for Scheme 5.1.1 in Theorem 5.1.5.Further combining Theorem 5.1.2,we obtain the following convergence error estimate of Scheme 5.1.1.Theorem 5.1.8.Let Assumptions 5.1.1,5.1.2 and 5.1.3 hold.Let the terrminal values of Scheme 5.1.1 satisfy where C is a constont independent of πN.If the coefficients of Scheme 5.1.1 simultaneously satisfy the consistency condition（0.28）and the stability condition（0.29）,then Scheme 5.1.1 is convergent with order r,that is,where（Yn,Zn）is given by Scheme 5.1.1.Based on Theorem 5.1.5 and the SSP multistep schemes[35-37,54,72,73]for ODEs,we propose the following SSPM schemes for FBSDEs.Scheme 5.2.1.（SSPM schemes for FBSDEs）A SSPM scheme for FBSDEs is the Scheme 5.1.1 with its coefficients satisfying where ε0 ∈(0,1]is a constant independent of πN.Theorem 5.1.5 and 5.1.8 imply Corollary 5.2.2,which gives the stability result and convergence error estimate of SSPM schemes for FBSDEs.Based on the SSP multistep schemes[35-37,54,72,73]for ODEs and Theorem 5.1.5,we respectively define the first and the second SSP coefficients for an explicit SSPM schemes for FBSDEs by Then the coefficients of the optimal k-step rth-order SSPM schemes for FBSDEs are given by where In Tables 5.1 and 5.2,we present some optimal SSPM schemes for FBSDEs under uniform time partitions.Chapter 6 Deep Learning Methods for FBSDEsIn this chapter,we will extend Scheme 5.1.1 to solve high-dimensional FBSDEs,where the key issue is to approximate the high-dimensional functions Yⁿ（x）:=E_tn^x[Yⁿ]和Zⁿ（x）:=E_tn^x[Zⁿ],and the high-dimensional conditional expectations Etnx[·].For n=0,1,…,N-k,we introduce probability densities pn defined on R^d,and then generate the set Dn of training points by where the elements xin in Dn are called the training points.Let yθ:R^d→R^p和Zλ:R^d→R^p×q be deep neural networks with parameters θ and λ,respectively.Then the approximations to the functions（Yn,Zn）are given by where with#Dn the number of elements in Dn.Based on Scheme 5.1.1,we apply the approximations in（0.32）and the Monte-Carlo methods with antithetic variables,leading to the following deep learning multistep schemes for FBSDEs.Scheme 6.1.2（Deep learning multistep schemes）Let yθ:R^d→RP and Zλ:R^d→R^p×q be deep neural networks with parameters θ and λ,respectively.For n=0,1,…,N-k and i=0,1,…,k-1,given the probability density functions pn and the terminal functions（YN-i,ZN-i）,for n=N-k,N-k-1,…,0,solve（Yn,Zn）by1.Obtain the training points by random sampling x1n,x2n,…,xmn,pn.Let2.For i=1,2,…,#Dn,solve（yiz,zin）by where Fⁿ⁺ⁱ（x）:=f(t_n+i,x,y_θn+i（x）,Z_λn+i（x）),x ∈ R^d,and Etnxin[·]is the Monte-Carlo approximation of Etnxin[·]given by（6.12）.3.Solve the following optimization problems...