An overview on deep learning-based approximation methods for partial differential equations

Christian Beck; Martin Hutzenthaler; Arnulf Jentzen; Benno Kuckuck

doi:10.3934/dcdsb.2022238

Article Contents

2023, Volume 28, Issue 6: 3697-3746. Doi: 10.3934/dcdsb.2022238

This issue Previous Article Finite-time flocking of Cucker-Smale model with unknown intrinsic dynamics Next Article The environmental effect on dynamics of the competition model with herd behavior

An overview on deep learning-based approximation methods for partial differential equations

1.
Department of Mathematics, ETH Zurich, 8092 Zurich, Switzerland
2.
Faculty of Mathematics, University of Duisburg-Essen, 45117 Essen, Germany
3.
School of Data Science and Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen, Shenzhen 518172, China
4.
Applied Mathematics: Institute for Analysis and Numerics, Faculty of Mathematics and Computer Science, University of Münster, 48149 Münster, Germany

^* Corresponding author: Benno Kuckuck
^* Corresponding author: Benno Kuckuck

Received: March 2021

Revised: November 2022

Early access: December 2022

Published: June 2023

The second author is supported by DFG grant HU1889/7-1.

Abstract / Introduction Full Text(HTML) Figure(1) / Table(9) Related Papers Cited by

Abstract

It is one of the most challenging problems in applied mathematics to approximatively solve high-dimensional partial differential equations (PDEs). Recently, several deep learning-based approximation algorithms for attacking this problem have been proposed and tested numerically on a number of examples of high-dimensional PDEs. This has given rise to a lively field of research in which deep learning-based methods and related Monte Carlo methods are applied to the approximation of high-dimensional PDEs. In this article we offer an introduction to this field of research by revisiting selected mathematical results related to deep learning approximation methods for PDEs and reviewing the main ideas of their proofs. We also provide a short overview of the recent literature in this area of research.

Keywords:

Mathematics Subject Classification: Primary: 65-02, 68T07; Secondary: 35-02, 65M99.

Citation:

Full Text(HTML)

Figure 1. Graphical illustration of a fully-connected feedforward artificial neural network consisting of $ L\in \mathbb N $ affine linear transformations (i.e., consisting of $ L+1 $ layers: one input layer, $ L-1 $ hidden layers, and one output layer) with $ l_0\in \mathbb N $ neurons on the input layer, with $ l_1\in \mathbb N $ neurons on the first hidden layer, with $ l_2\in \mathbb N $ neurons on the second hidden layer, $ \dots $, with $ l_{L-1} $ neurons on the $ (L-1) $-th hidden layer, and with $ l_L $ neurons in the output layer

Download: Full-size image PowerPoint slide

Table 1. The hyperparameters used in the training for the first and third simulation using the deep Galerkin method (see the results in Tables 3 and 5)

$d$	Batch size	Hidden layers	Neurons per hidden layer	Steps	Initial learning rate	Learning rate decay
$1$	$256$	$3$	$41$	$350$	$0.05$	$0.98$
$2$	$256$	$3$	$42$	$350$	$0.05$	$0.98$
$5$	$256$	$3$	$45$	$350$	$0.05$	$0.98$
$10$	$256$	$3$	$50$	$350$	$0.05$	$0.98$
$20$	$256$	$3$	$60$	$500$	$0.025$	$0.99$
$50$	$256$	$3$	$90$	$750$	$0.01$	$0.995$
$100$	$256$	$3$	$140$	$750$	$0.01$	$0.995$
$200$	$256$	$3$	$240$	$750$	$0.005$	$0.995$

| Show Table

DownLoad: CSV

Table 2. The hyperparameters used in the training for the second simulation using the deep Galerkin method (see the results in Table 4)

$d$	Batch size	Hidden layers	Neurons per hidden layer	Steps	Initial learning rate	Learning rate decay
$1$	$256$	$3$	$41$	$500$	$0.05$	$0.99$
$2$	$256$	$3$	$42$	$500$	$0.05$	$0.99$
$5$	$256$	$3$	$45$	$500$	$0.05$	$0.99$
$10$	$256$	$3$	$50$	$750$	$0.025$	$0.995$
$20$	$256$	$3$	$60$	$750$	$0.025$	$0.995$
$50$	$256$	$3$	$90$	$1000$	$0.01$	$0.998$
$100$	$256$	$3$	$140$	$1000$	$0.01$	$0.998$
$200$	$256$	$3$	$240$	$1000$	$0.005$	$0.998$

| Show Table

DownLoad: CSV

Table 3. Approximations for $u(1/2, 0, 0, \dots, 0)$ where $u$ is the solution of the PDE in (35) with the initial value $ \mathbb R^d\ni x\mapsto u(0, x) = \sqrt{1+||x||^2}\in \mathbb R$ for $d\in\{1, 2, 5, 10, 20, 50, 100, 200\}$ using the deep Galerkin method. For the hyperparameters used in the training, see Table 1. For the Python source code used to obtain these results, see Section 8.2

$d$	Mean	Standard deviation	Reference value	Absolute $L^1$-error	Relative $L^1$-error	Average runtime
$1$	$1.788665$	$0.013821$	$1.836708$	$0.048043$	$0.026157$	$2.48$
$2$	$2.048171$	$0.025434$	$2.109239$	$0.061302$	$0.029064$	$3.47$
$5$	$2.684484$	$0.071031$	$2.648632$	$0.057190$	$0.021592$	$6.76$
$10$	$3.171959$	$0.027321$	$3.208623$	$0.039949$	$0.012451$	$17.84$
$20$	$4.041396$	$0.112243$	$4.107034$	$0.088848$	$0.021633$	$34.38$
$50$	$7.193619$	$0.120224$	$7.490964$	$0.297344$	$0.039694$	$133.41$
$100$	$9.979369$	$0.409066$	$9.808476$	$0.372600$	$0.037988$	$292.10$
$200$	$13.430278$	$0.247504$	$14.604635$	$1.174357$	$0.080410$	$673.06$

| Show Table

DownLoad: CSV

Table 4. Approximations for $u(1/2, 0, 0, \dots, 0)$ where $u$ is the solution of the PDE in (35) with the initial value $ \mathbb R^d\ni x\mapsto u(0, x) = 2/(4+||x||^2)\in \mathbb R$ for $d\in\{1, 2, 5, 10, 20, 50, 100, 200\}$ using the deep Galerkin method. For the hyperparameters used in the training, see Table 2. For the Python source code used to obtain these results, see Section 8.2

$d$	Mean	Standard deviation	Reference value	Absolute $L^1$-error	Relative $L^1$-error	Average runtime
$1$	$0.680122$	$0.003512$	$0.677511$	$0.003552$	$0.005243$	$3.62$
$2$	$0.590964$	$0.009817$	$0.584510$	$0.010066$	$0.017221$	$4.87$
$5$	$0.374912$	$0.033546$	$0.404080$	$0.030547$	$0.075597$	$9.65$
$10$	$0.281780$	$0.014043$	$0.258967$	$0.025007$	$0.096566$	$26.19$
$20$	$0.139796$	$0.013930$	$0.147140$	$0.013351$	$0.090739$	$49.88$
$50$	$0.076818$	$0.005612$	$0.063228$	$0.013589$	$0.214926$	$175.57$
$100$	$0.043529$	$0.008632$	$0.032302$	$0.011817$	$0.365843$	$384.41$
$200$	$0.020501$	$0.014760$	$0.016318$	$0.012173$	$0.745950$	$895.37$

| Show Table

DownLoad: CSV

Table 5. Approximations for $u(1/2, 0, 0, \dots, 0)$ where $u$ is the solution of the PDE in (35) with the initial value $ \mathbb R^d\ni x\mapsto u(0, x) = \arctan\bigl(\tfrac{||x||}2\bigr)\in \mathbb R$ for $d\in\{1, 2, 5, 10, 20, 50, 100, 200\}$ using the deep Galerkin method. For the hyperparameters used in the training of the DNNs, see Table 1. For the Python source code used to obtain these results, see Section 8.2

$d$	Mean	Standard deviation	Reference value	Absolute $L^1$-error	Relative $L^1$-error	Average runtime
$1$	$0.629165$	$0.029052$	$0.569925$	$0.061155$	$0.107303$	$2.47$
$2$	$0.900790$	$0.016736$	$0.835350$	$0.065440$	$0.078338$	$3.52$
$5$	$1.249522$	$0.020660$	$1.201798$	$0.047724$	$0.039711$	$6.77$
$10$	$1.395483$	$0.022569$	$1.440293$	$0.044975$	$0.031226$	$18.29$
$20$	$1.593871$	$0.029176$	$1.622937$	$0.035450$	$0.021843$	$34.28$
$50$	$1.789839$	$0.032926$	$1.785912$	$0.027953$	$0.015652$	$131.69$
$100$	$1.851764$	$0.034147$	$1.866269$	$0.030308$	$0.016240$	$291.67$
$200$	$1.626139$	$0.206072$	$1.921878$	$0.295739$	$0.153880$	$678.89$

| Show Table

DownLoad: CSV

Table 6. The hyperparameters used in the training for the simulations using the deep splitting method (see the results in Tables 7–9)

$d$	Batch size	Hidden layers	Neurons per hidden layer	Steps	Initial learning rate	Learning rate decay
$1$	$256$	$2$	$51$	$350$	$0.2$	$0.985$
$2$	$256$	$2$	$52$	$350$	$0.2$	$0.985$
$5$	$256$	$2$	$55$	$350$	$0.2$	$0.985$
$10$	$256$	$2$	$60$	$350$	$0.2$	$0.985$
$20$	$256$	$2$	$70$	$500$	$0.2$	$0.99$
$50$	$256$	$2$	$100$	$500$	$0.2$	$0.99$
$100$	$256$	$2$	$150$	$750$	$0.2$	$0.995$
$200$	$256$	$2$	$250$	$1000$	$0.2$	$0.995$
$500$	$256$	$2$	$550$	$1000$	$0.2$	$0.995$
$1000$	$256$	$2$	$1050$	$1250$	$0.2$	$0.998$

| Show Table

DownLoad: CSV

Table 7. Approximations for $u(1/2, 0, 0, \dots, 0)$ where $u$ is the solution of the PDE in (35) with the initial value $ \mathbb R^d\ni x\mapsto u(0, x) = \sqrt{1+||x||^2}\in \mathbb R$ for $d\in\{1, 2, 5, 10, 20, 50, 100, 200, 500, 1000\}$ using the deep splitting method. For the hyperparameters used in the training of the DNNs, see Table 6. For the Python source code used to obtain these results, see Section 8.3

$d$	Mean	Standard deviation	Reference value	Absolute $L^1$-error	Relative $L^1$-error	Average runtime
$1$	$1.830958$	$0.015773$	$1.836708$	$0.013452$	$0.007324$	$20.22$
$2$	$2.108470$	$0.014779$	$2.109239$	$0.012182$	$0.005776$	$19.90$
$5$	$2.642265$	$0.015791$	$2.648632$	$0.014881$	$0.005618$	$20.13$
$10$	$3.196520$	$0.016279$	$3.208623$	$0.016151$	$0.005034$	$20.31$
$20$	$4.082424$	$0.019874$	$4.107034$	$0.026197$	$0.006379$	$28.89$
$50$	$7.480304$	$0.027910$	$7.490964$	$0.022157$	$0.002958$	$29.73$
$100$	$9.804550$	$0.009534$	$9.808476$	$0.008295$	$0.000846$	$46.51$
$200$	$14.602816$	$0.036716$	$14.604635$	$0.020845$	$0.001427$	$64.86$
$500$	$21.386109$	$1.045499$	$22.230142$	$0.844033$	$0.037968$	$75.23$
$1000$	$31.748045$	$0.126012$	$31.751259$	$0.068201$	$0.002148$	$117.77$

| Show Table

DownLoad: CSV

Table 8. Approximations for $u(1/2, 0, 0, \dots, 0)$ where $u$ is the solution of the PDE in (35) with the initial value $ \mathbb R^d\ni x\mapsto u(0, x) = 2/(4+||x||^2)\in \mathbb R$ for $d\in\{1, 2, 5, 10, 20, 50, 100, 200, 500, 1000\}$ using the deep splitting method. For the hyperparameters used in the training of the DNNs, see Table 6. For the Python source code used to obtain these results, see Section 8.3

$d$	Mean	Standard deviation	Reference value	Absolute $L^1$-error	Relative $L^1$-error	Average runtime
$1$	$0.678383$	$0.004441$	$0.677511$	$0.003652$	$0.005390$	$20.24$
$2$	$0.583808$	$0.005164$	$0.584510$	$0.004229$	$0.007235$	$19.93$
$5$	$0.405637$	$0.003722$	$0.404080$	$0.003680$	$0.009107$	$20.08$
$10$	$0.257737$	$0.001830$	$0.258967$	$0.001949$	$0.007524$	$20.33$
$20$	$0.146415$	$0.000674$	$0.147140$	$0.000859$	$0.005841$	$29.08$
$50$	$0.062905$	$0.000134$	$0.063228$	$0.000323$	$0.005114$	$30.05$
$100$	$0.032181$	$0.000150$	$0.032302$	$0.000146$	$0.004514$	$46.48$
$200$	$0.016254$	$0.000023$	$0.016318$	$0.000064$	$0.003941$	$64.93$
$500$	$0.006542$	$0.000005$	$0.006568$	$0.000026$	$0.003909$	$75.37$
$1000$	$0.003277$	$0.000010$	$0.003291$	$0.000015$	$0.004467$	$118.31$

| Show Table

DownLoad: CSV

Table 9. Approximations for $u(1/2, 0, 0, \dots, 0)$ where $u$ is the solution of the PDE in (35) with the initial value $ \mathbb R^d\ni x\mapsto u(0, x) = \arctan\bigl(\tfrac{||x||}2\bigr)\in \mathbb R$ for $d\in\{1, 2, 5, 10, 20, 50, 100, 200, 500, 1000\}$ using the deep splitting method. For the hyperparameters used in the training of the DNNs, see Table 6. For the Python source code used to obtain these results, see Section 8.3

$d$	Mean	Standard deviation	Reference value	Absolute $L^1$-error	Relative $L^1$-error	Average runtime
$1$	$0.561503$	$0.012954$	$0.569925$	$0.012631$	$0.022162$	$20.16$
$2$	$0.830821$	$0.011572$	$0.835350$	$0.009811$	$0.011745$	$20.19$
$5$	$1.195350$	$0.007251$	$1.201798$	$0.008002$	$0.006658$	$20.09$
$10$	$1.436600$	$0.003500$	$1.440293$	$0.004196$	$0.002913$	$20.22$
$20$	$1.622351$	$0.001493$	$1.622937$	$0.001305$	$0.000804$	$29.17$
$50$	$1.785940$	$0.000271$	$1.785912$	$0.000214$	$0.000120$	$29.81$
$100$	$1.866438$	$0.000438$	$1.866269$	$0.000337$	$0.000181$	$46.65$
$200$	$1.922286$	$0.000091$	$1.921878$	$0.000408$	$0.000212$	$65.01$
$500$	$1.970859$	$0.000031$	$1.970294$	$0.000566$	$0.000287$	$75.16$
$1000$	$1.994976$	$0.000086$	$1.994330$	$0.000645$	$0.000324$	$118.21$

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	L. Abbas-Turki, B. Diallo and G. Pagès, Conditional Monte Carlo Learning for Diffusions I: main methodology and application to backward stochastic differential equations, Preprint available at https://hal.archives-ouvertes.fr/hal-02959492, 28 pages.
[2]	L. Abbas-Turki, B. Diallo and G. Pagès, Conditional Monte Carlo Learning for Diffusions II: extended methodology and application to risk measures and early stopping problems, Preprint available at https://hal.archives-ouvertes.fr/hal-02959494, 31 pages.
[3]	A. Al-Aradi, A. Correia, G. Jardim, D. de Freitas Naiff and Y. Saporito, Extensions of the deep Galerkin method, Appl. Math. Comput., 430 (2022), 127287. doi: 10.1016/j.amc.2022.127287.
[4]	K. Andersson, A. Andersson and C. W. Oosterlee, Convergence of a robust deep FBSDE method for stochastic control, arXiv: 2201.06854, 27 pages.
[5]	C. Anitescu, E. Atroshchenko, N. Alajlan and T. Rabczuk, Artificial neural network methods for the solution of second order boundary value problems, Comput. Mater. Contin., 59 (2019), 345-359. doi: 10.32604/cmc.2019.06641.
[6]	V. Bally and G. Pagès, Error analysis of the optimal quantization algorithm for obstacle problems, Stochastic Process. Appl., 106 (2003), 1-40. doi: 10.1016/S0304-4149(03)00026-7.
[7]	V. Bally and G. Pagès, A quantization algorithm for solving multi-dimensional discrete-time optimal stopping problems, Bernoulli, 9 (2003), 1003-1049. doi: 10.3150/bj/1072215199.
[8]	C. Bayer, J. Qiu and Y. Yao, Pricing options under rough volatility with backward SPDEs, SIAM J. Financial Math., 13 (2022), 179-212. doi: 10.1137/20M1357639.
[9]	C. Beck, S. Becker, P. Cheridito, A. Jentzen and A. Neufeld, Deep learning based numerical approximation algorithms for stochastic partial differential equations and high-dimensional nonlinear filtering problems, arXiv: 2012.01194, 58 pages.
[10]	C. Beck, S. Becker, P. Cheridito, A. Jentzen and A. Neufeld, Deep splitting method for parabolic PDEs, SIAM J. Sci. Comput., 43 (2021), A3135–A3154. doi: 10.1137/19M1297919.
[11]	C. Beck, S. Becker, P. Grohs, N. Jaafari and A. Jentzen, Solving the Kolmogorov PDE by means of deep learning, J. Sci. Comput., 88 (2021), Paper no. 73, 28 pp. doi: 10.1007/s10915-021-01590-0.
[12]	C. Beck, W. E and A. Jentzen, Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations, J. Nonlinear Sci., 29 (2019), 1563-1619. doi: 10.1007/s00332-018-9525-3.
[13]	C. Beck, L. Gonon and A. Jentzen, Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations, arXiv: 2003.00596, 50 pages.
[14]	C. Beck, F. Hornung, M. Hutzenthaler, A. Jentzen and T. Kruse, Overcoming the curse of dimensionality in the numerical approximation of Allen–Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations, J. Numer. Math., 28 (2020), 197-222. doi: 10.1515/jnma-2019-0074.
[15]	C. Beck, M. Hutzenthaler and A. Jentzen, On nonlinear Feynman–Kac formulas for viscosity solutions of semilinear parabolic partial differential equations, Stoch. Dyn., 21 (2021), Paper no. 2150048, 68 pp. doi: 10.1142/S0219493721500489.
[16]	C. Beck, A. Jentzen and T. Kruse, Nonlinear Monte Carlo methods with polynomial runtime for high-dimensional iterated nested expectations, arXiv: 2009.13989, 47 pages.
[17]	S. Becker, R. Braunwarth, M. Hutzenthaler, A. Jentzen and P. von Wurstemberger, Numerical simulations for full history recursive multilevel Picard approximations for systems of high-dimensional partial differential equations, Commun. Comput. Phys., 28 (2020), 2109-2138. doi: 10.4208/cicp.OA-2020-0130.
[18]	S. Becker, P. Cheridito and A. Jentzen, Deep optimal stopping, J. Mach. Learn. Res., 20 (2019), 1-25.
[19]	S. Becker, P. Cheridito, A. Jentzen and T. Welti, Solving high-dimensional optimal stopping problems using deep learning, European J. Appl. Math., 32 (2021), 470-514. doi: 10.1017/S0956792521000073.
[20]	S. Becker, A. Jentzen, M. S. Müller and P. von Wurstemberger, Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing, arXiv: 2002.02717, 70 pages.
[21]	C. Bender and R. Denk, A forward scheme for backward SDEs, Stochastic Process. Appl., 117 (2007), 1793-1812. doi: 10.1016/j.spa.2007.03.005.
[22]	C. Bender and J. Steiner, Least-squares Monte Carlo for backward SDEs, in Numerical methods in finance, Springer, 2012,257–289. doi: 10.1007/978-3-642-25746-9_8.
[23]	C. Bender and J. Zhang, Time discretization and Markovian iteration for coupled FBSDEs, Ann. Appl. Probab., 18 (2008), 143-177. doi: 10.1214/07-AAP448.
[24]	P. Beneventano, P. Cheridito, A. Jentzen and P. von Wurstemberger, High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations, arXiv: 2012.04326, 32 pages.
[25]	J. Berg and K. Nyström, A unified deep artificial neural network approach to partial differential equations in complex geometries, Neurocomputing, 317 (2018), 28-41. doi: 10.1016/j.neucom.2018.06.056.
[26]	J. Berner, P. Grohs and A. Jentzen, Analysis of the generalization error: Empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations, SIAM J. Math. Data Sci., 2 (2020), 631-657. doi: 10.1137/19M125649X.
[27]	S. Berrone, C. Canuto and M. Pintore, Solving PDEs by variational physics-informed neural networks: an a posteriori error analysis, Ann. Univ. Ferrara Sez. VII Sci. Mat., 68 (2022), 575-595. doi: 10.1007/s11565-022-00441-6.
[28]	S. Berrone, C. Canuto and M. Pintore, Variational physics informed neural networks: the role of quadratures and test functions, J. Sci. Comput., 92 (2022), 100. doi: 10.1007/s10915-022-01950-4.
[29]	J. Blechschmidt and O. G. Ernst, Three ways to solve partial differential equations with neural networks-a review, GAMM-Mitt., 44 (2021), e202100006. doi: 10.1002/gamm.202100006.
[30]	B. Bouchard and N. Touzi, Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations, Stochastic Process. Appl., 111 (2004), 175-206. doi: 10.1016/j.spa.2004.01.001.
[31]	V. Boussange, S. Becker, A. Jentzen, B. Kuckuck and L. Pellissier, Deep learning approximations for non-local nonlinear PDEs with Neumann boundary conditions, arXiv: 2205.03672, 59 pages.
[32]	J. Brandstetter, R. van den Berg, M. Welling and J. K. Gupta, Clifford neural layers for PDE modeling, arXiv: 2209.04934, 52 pages.
[33]	P. Briand, B. Delyon and J. Mémin, Donsker-type theorem for BSDEs, Electron. Commun. Probab., 6 (2001), 1-14. doi: 10.1214/ECP.v6-1030.
[34]	P. Briand and C. Labart, Simulation of BSDEs by Wiener chaos expansion, Ann. Appl. Probab., 24 (2014), 1129-1171. doi: 10.1214/13-AAP943.
[35]	S. Cai, Z. Mao, Z. Wang, M. Yin and G. E. Karniadakis, Physics-informed neural networks (PINNs) for fluid mechanics: A review, Acta Mech. Sin., 37 (2021), 1727-1738. doi: 10.1007/s10409-021-01148-1.
[36]	S. Cai, Z. Wang, S. Wang, P. Perdikaris and G. E. Karniadakis, Physics-Informed Neural Networks for Heat Transfer Problems, J. Heat Transfer, 143, 060801. doi: 10.1115/1.4050542.
[37]	Z. Cai and J. Liu, Approximating quantum many-body wave functions using artificial neural networks, Phys. Rev. B, 97 (2018), 035116. doi: 10.1103/PhysRevB.97.035116.
[38]	G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science, 355 (2017), 602-606. doi: 10.1126/science.aag2302.
[39]	R. Carmona and M. Laurière, Convergence Analysis of Machine Learning Algorithms for the Numerical Solution of Mean Field Control and Games: II – The Finite Horizon Case, arXiv: 1908.01613, 39 pages.
[40]	R. Carmona and M. Laurière, Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games I: The ergodic case, SIAM J. Numer. Anal., 59 (2021), 1455-1485. doi: 10.1137/19M1274377.
[41]	J. Castro, Deep learning schemes for parabolic nonlocal integro-differential equations, Partial Differ. Equ. Appl., 3 (2022), 77. doi: 10.1007/s42985-022-00213-z.
[42]	Q. Chan-Wai-Nam, J. Mikael and X. Warin, Machine learning for semi linear PDEs, J. Sci. Comput., 79 (2019), 1667-1712. doi: 10.1007/s10915-019-00908-3.
[43]	D. Chang, H. Liu and J. Xiong, A branching particle system approximation for a class of FBSDEs, Prob. Uncertain. Quant. Risk, 1 (2016), 1-34. doi: 10.1186/s41546-016-0007-y.
[44]	J.-F. Chassagneux and C. Garcia Trillos, Cubature method to solve BSDEs: Error expansion and complexity control, Math. Comput., 89 (2020), 1895-1932. doi: 10.1090/mcom/3522.
[45]	J.-F. Chassagneux and A. Richou, Numerical stability analysis of the Euler scheme for BSDEs, SIAM J. Numer. Anal., 53 (2015), 1172-1193. doi: 10.1137/140977047.
[46]	F. Chen, D. Sondak, P. Protopapas, M. Mattheakis, S. Liu, D. Agarwal and M. Di Giovanni, NeuroDiffEq: A Python package for solving differential equations with neural networks, J. Open Source Softw., 5 (2020), 1931. doi: 10.21105/joss.01931.
[47]	Y. Chen and J. W. L. Wan, Deep neural network framework based on backward stochastic differential equations for pricing and hedging american options in high dimensions, Quant. Finance, 21 (2021), 45-67. doi: 10.1080/14697688.2020.1788219.
[48]	Y. Chen, L. Lu, G. E. Karniadakis and L. Dal Negro, Physics-informed neural networks for inverse problems in nano-optics and metamaterials, Opt. Express, 28 (2020), 11618-11633. doi: 10.4208/cicp.OA-2020-0164.
[49]	P. Cheridito and M. Stadje, BS$\Delta$Es and BSDEs with non-Lipschitz drivers: comparison, convergence and robustness, Bernoulli, 19 (2013), 1047-1085. doi: 10.3150/12-BEJ445.
[50]	D.-A. Clevert, T. Unterthiner and S. Hochreiter, Fast and accurate deep network learning by exponential linear units (ELUs), arXiv: 1511.07289, 14 pages.
[51]	S. Cox and J. van Neerven, Pathwise Hölder convergence of the implicit-linear Euler scheme for semi-linear SPDEs with multiplicative noise, Numer. Math., 125 (2013), 259-345. doi: 10.1007/s00211-013-0538-4.
[52]	D. Crisan and K. Manolarakis, Solving backward stochastic differential equations using the cubature method: Application to nonlinear pricing, SIAM J. Financial Math., 3 (2012), 534-571. doi: 10.1137/090765766.
[53]	D. Crisan and K. Manolarakis, Second order discretization of backward SDEs and simulation with the cubature method, Ann. Appl. Probab., 24 (2014), 652-678. doi: 10.1214/13-AAP932.
[54]	D. Crisan, K. Manolarakis and N. Touzi, On the Monte Carlo simulation of BSDEs: An improvement on the {M}alliavin weights, Stochastic Process. Appl., 120 (2010), 1133-1158. doi: 10.1016/j.spa.2010.03.015.
[55]	S. Cuomo, V. Schiano di Cola, F. Giampaolo, G. Rozza, M. Raissi and F. Piccialli, Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next, J. Sci. Comp., 92 (2022), 88. doi: 10.1007/s10915-022-01939-z.
[56]	J. Darbon, G. P. Langlois and T. Meng, Overcoming the curse of dimensionality for some Hamilton-Jacobi partial differential equations via neural network architectures, Res. Math. Sci., 7 (2020), Paper No. 20. doi: 10.1007/s40687-020-00215-6.
[57]	J. Darbon and T. Meng, On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton-Jacobi partial differential equations, J. Comput. Phys., 425 (2021), 109907. doi: 10.1016/j.jcp.2020.109907.
[58]	J. Darbon and S. Osher, Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere, Res. Math. Sci., 3. doi: 10.1186/s40687-016-0068-7.
[59]	P. C. de Raynal and C. G. Trillos, A cubature based algorithm to solve decoupled McKean–Vlasov forward–backward stochastic differential equations, Stochastic Process. Appl., 125 (2015), 2206-2255. doi: 10.1016/j.spa.2014.11.018.
[60]	F. Delarue and S. Menozzi, A forward-backward stochastic algorithm for quasi-linear PDEs, Ann. Appl. Probab., 140–184. doi: 10.1214/105051605000000674.
[61]	F. Delarue and S. Menozzi, An interpolated stochastic algorithm for quasi-linear PDEs, Math. Comput., 77 (2008), 125-158. doi: 10.1090/S0025-5718-07-02008-X.
[62]	T. Deveney, E. Mueller and T. Shardlow, A deep surrogate approach to efficient Bayesian inversion in PDE and integral equation models, arXiv: 1910.01547, 29 pages.
[63]	M. W. M. G. Dissanayake and N. Phan-Thien, Neural-network-based approximations for solving partial differential equations, Commun. Numer. Methods Engrg., 10 (1994), 195-201. doi: 10.1002/cnm.1640100303.
[64]	T. Dockhorn, A discussion on solving partial differential equations using neural networks, arXiv: 1904.07200, 9 pages.
[65]	S. Dolgov, D. Kalise and K. K. Kunisch, Tensor decomposition methods for high-dimensional Hamilton–Jacobi–Bellman equations, SIAM J. Sci. Comput., 43 (2021), A1625–A1650. doi: 10.1137/19M1305136.
[66]	W. E, J. Han and A. Jentzen, Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations, Commun. Math. Stat., 5 (2017), 349-380. doi: 10.1007/s40304-017-0117-6.
[67]	W. E, J. Han and A. Jentzen, Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning, Nonlinearity, 35 (2022), 278-310. doi: 10.1088/1361-6544/ac337f.
[68]	W. E, M. Hutzenthaler, A. Jentzen and T. Kruse, On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations, J. Sci. Comput., 79 (2019), 1534-1571. doi: 10.1007/s10915-018-00903-0.
[69]	W. E, M. Hutzenthaler, A. Jentzen and T. Kruse, Multilevel Picard iterations for solving smooth semilinear parabolic heat equations, Partial Differ. Equ. Appl., 2 (2021), Paper no. 80. doi: 10.1007/s42985-021-00089-5.
[70]	W. E and B. Yu, The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat., 6 (2018), 1-12. doi: 10.1007/s40304-018-0127-z.
[71]	M. Eigel, M. Pfeffer and R. Schneider, Adaptive stochastic Galerkin FEM with hierarchical tensor representations, Numer. Math., 136 (2017), 765-803. doi: 10.1007/s00211-016-0850-x.
[72]	D. Elbrächter, P. Grohs, A. Jentzen and C. Schwab, DNN Expression Rate Analysis of High-dimensional PDEs: Application to Option Pricing, Constr. Approx., 55 (2022), 3-71. doi: 10.1007/s00365-021-09541-6.
[73]	Y. Fan and L. Ying, Solving electrical impedance tomography with deep learning, J. Comput. Phys., 404 (2020), 109119. doi: 10.1016/j.jcp.2019.109119.
[74]	Y. Fan and L. Ying, Solving inverse wave scattering with deep learning, Ann. Math. Sci. Appl., 7 (2022), 23-48. doi: 10.4310/AMSA.2022.v7.n1.a2.
[75]	R. Frey and V. Köck, Deep neural network algorithms for parabolic PIDEs and applications in insurance mathematics, in Mathematical and Statistical Methods for Actuarial Sciences and Finance (eds. M. Corazza, C. Perna, C. Pizzi and M. Sibillo), Springer International Publishing, Cham, 2022,272–277. doi: 10.1007/978-3-030-99638-3_44.
[76]	Y. Fu, W. Zhao and T. Zhou, Multistep schemes for forward backward stochastic differential equations with jumps, J. Sci. Comput., 69 (2016), 651-672. doi: 10.1007/s10915-016-0212-y.
[77]	M. Fujii, A. Takahashi and M. Takahashi, Asymptotic expansion as prior knowledge in deep learning method for high dimensional BSDEs, Asia-Pac. Financ. Markets, 26 (2019), 391-408.
[78]	C. Geiss, C. Labart and A. Luoto, Mean square rate of convergence for random walk approximation of forward-backward SDEs, Adv. in Appl. Probab., 52 (2020), 735-771. doi: 10.1017/apr.2020.17.
[79]	C. Geiss, C. Labart and A. Luoto, Random walk approximation of BSDEs with Hölder continuous terminal condition, Bernoulli, 26 (2020), 159-190. doi: 10.3150/19-BEJ1120.
[80]	C. Geiss and C. Labart, Simulation of BSDEs with jumps by Wiener Chaos expansion, Stochastic Process. Appl., 126 (2016), 2123-2162. doi: 10.1016/j.spa.2016.01.006.
[81]	M. Geist, P. Petersen, M. Raslan, R. Schneider and G. Kutyniok, Numerical solution of the parametric diffusion equation by deep neural networks, J. Sci. Comput., 88 (2021), Paper No. 22, 37. doi: 10.1007/s10915-021-01532-w.
[82]	M. Germain, H. Pham and X. Warin, Approximation error analysis of some deep backward schemes for nonlinear PDEs, SIAM J. Sci. Comput., 44 (2022), A28–A56. doi: 10.1137/20M1355355.
[83]	M. B. Giles, Multilevel Monte Carlo path simulation, Oper. Res., 56 (2008), 607-617. doi: 10.1287/opre.1070.0496.
[84]	M. B. Giles, Multilevel Monte Carlo methods, Acta Numer., 24 (2015), 259-328. doi: 10.1017/S096249291500001X.
[85]	M. B. Giles, A. Jentzen and T. Welti, Generalised multilevel Picard approximations, arXiv: 1911.03188, 61 pages.
[86]	X. Glorot and Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, 2010,249–256.
[87]	E. Gobet and C. Labart, Solving BSDE with adaptive control variate, SIAM J. Numer. Anal., 48 (2010), 257-277. doi: 10.1137/090755060.
[88]	E. Gobet and J.-P. Lemor, Numerical simulation of BSDEs using empirical regression methods: theory and practice, in 5th Colloquium on BSDEs (eds. S. Tang and S. Peng), Shanghai, Chine, 2006.
[89]	E. Gobet, J.-P. Lemor and X. Warin, A regression-based Monte Carlo method to solve backward stochastic differential equations, Ann. Appl. Probab., 15 (2005), 2172-2202. doi: 10.1214/105051605000000412.
[90]	E. Gobet, J. G. López-Salas, P. Turkedjiev and C. Vázquez, Stratified regression Monte-Carlo scheme for semilinear PDEs and BSDEs with large scale parallelization on GPUs, SIAM J. Sci. Comput., 38 (2016), C652–C677. doi: 10.1137/16M106371X.
[91]	E. Gobet and P. Turkedjiev, Approximation of backward stochastic differential equations using {M}alliavin weights and least-squares regression, Bernoulli, 22 (2016), 530-562. doi: 10.3150/14-BEJ667.
[92]	E. Gobet and P. Turkedjiev, Linear regression MDP scheme for discrete backward stochastic differential equations under general conditions, Math. Comput., 85 (2016), 1359-1391. doi: 10.1090/mcom/3013.
[93]	L. Gonon and C. Schwab, Deep ReLU network expression rates for option prices in high-dimensional, exponential Lévy models, Technical Report 2020-52, Seminar for Applied Mathematics, ETH Zürich, Switzerland, 2020. doi: 10.1007/s00780-021-00462-7.
[94]	L. Gonon, Random feature neural networks learn Black–Scholes type PDEs without curse of dimensionality, arXiv: 2106.08900, 46 pages.
[95]	L. Gonon, P. Grohs, A. Jentzen, D. Kofler and D. Ši\|s\|ka, Uniform error estimates for artificial neural network approximations for heat equations, IMA J. Numer. Anal., 64 pp. doi: 10.1093/imanum/drab027.
[96]	L. Gonon, J. Muhle-Karbe and X. Shi, Asset pricing with general transaction costs: theory and numerics, Math. Finance, 31 (2021), 595-648. doi: 10.1111/mafi.12297.
[97]	V. I. Gorikhovskii, T. O. Evdokimova and V. A. Poletansky, Neural networks in solving differential equations, J. Phys. Conf. Ser., 2308 (2022), 012008. doi: 10.1088/1742-6596/2308/1/012008.
[98]	L. Goudenège, A. Molent and A. Zanette, Variance reduction applied to machine learning for pricing Bermudan/American options in high dimension, arXiv: 1903.11275, 25 pages.
[99]	P. Grohs and L. Herrmann, Deep neural network approximation for high-dimensional elliptic PDEs with boundary conditions, IMA J. Numer. Anal., 42 (2022), 2055-2082. doi: 10.1093/imanum/drab031.
[100]	P. Grohs, F. Hornung, A. Jentzen and P. Von Wurstemberger, A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations, To appear in Mem. Amer. Math. Soc., arXiv: 1809.02362,124 pages.
[101]	P. Grohs, A. Jentzen and D. Salimova, Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms, Partial Differ. Equ. Appl., 3 (2022), 45, 41 pp. doi: 10.1007/s42985-021-00100-z.
[102]	L. Guo, H. Wu, X. Yu and T. Zhou, Monte Carlo fPINNs: Deep learning method for forward and inverse problems involving high dimensional fractional partial differential equations, Comput. Methods Appl. Mech. Engrg., 400 (2022), 115523. doi: 10.1016/j.cma.2022.115523.
[103]	I. Gyöngy and N. Krylov, On the splitting-up method and stochastic partial differential equations, Ann. Probab., 31 (2003), 564-591. doi: 10.1214/aop/1048516528.
[104]	B. Güler, A. Laignelet and P. Parpas, Towards Robust and Stable Deep Learning Algorithms for Forward Backward Stochastic Differential Equations, arXiv: 1910.11623, 10 pages.
[105]	E. Haghighat, A. C. Bekar, E. Madenci and R. Juanes, A nonlocal physics-informed deep learning framework using the peridynamic differential operator, Comput. Methods Appl. Mech. Engrg., 385. doi: 10.1016/j.cma.2021.114012.
[106]	M. Hairer, M. Hutzenthaler and A. Jentzen, Loss of regularity for Kolmogorov equations, Ann. Probab., 43 (2015), 468-527. doi: 10.1214/13-AOP838.
[107]	J. Han and R. Hu, Deep fictitious play for finding Markovian Nash equilibrium in multi-agent games, in Proceedings of The First Mathematical and Scientific Machine Learning Conference (MSML), vol. 107, 2020,221–245.
[108]	J. Han, R. Hu and J. Long, Convergence of deep fictitious play for stochastic differential games, Front. Math. Finance, 1 (2022), 287-319. doi: 10.3934/fmf.2021011.
[109]	J. Han, A. Jentzen and W. E, Solving high-dimensional partial differential equations using deep learning, Proc. Natl. Acad. Sci. USA, 115 (2018), 8505-8510. doi: 10.1073/pnas.1718942115.
[110]	J. Han and J. Long, Convergence of the deep BSDE method for coupled FBSDEs, Prob. Uncertain. Quant. Risk, 5. doi: 10.1186/s41546-020-00047-w.
[111]	J. Han, J. Lu and M. Zhou, Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion Monte Carlo like approach, J. Comput. Phys., 423 (2020), 109792. doi: 10.1016/j.jcp.2020.109792.
[112]	J. Han, L. Zhang and W. E, Solving many-electron Schrödinger equation using deep neural networks, J. Comput. Phys., 399 (2019), 108929. doi: 10.1016/j.jcp.2019.108929.
[113]	J. Han, M. Nica and A. R. Stinchcombe, A derivative-free method for solving elliptic partial differential equations with deep neural networks, J. Comput. Phys., 419 (2020), 109672. doi: 10.1016/j.jcp.2020.109672.
[114]	S. Heinrich, Monte Carlocomplexity of global solution of integral equations, J. Complex., 14 (1998), 151-175. doi: 10.1006/jcom.1998.0471.
[115]	S. Heinrich, Multilevel Monte Carlo methods, in International Conference on Large-Scale Scientific Computing, Springer, 2001, 58–67. doi: 10.1007/3-540-45346-6_5.
[116]	S. Heinrich and E. Sindambiwe, Monte Carlo complexity of parametric integration, J. Complex., 15 (1999), 317-341. doi: 10.1006/jcom.1999.0508.
[117]	P. Henry-Labordère, Counterparty risk valuation: A marked branching diffusion approach, arXiv: 1203.2369, 17 pages.
[118]	P. Henry-Labordère, Deep Primal-Dual Algorithm for BSDEs: Applications of Machine Learning to CVA and IM, Available at SSRN 3071506.
[119]	P. Henry-Labordère, N. Oudjane, X. Tan, N. Touzi and X. Warin, Branching diffusion representation of semilinear PDEs and Monte Carlo approximation, Ann. Inst. Henri Poincaré Probab. Stat., 55 (2019), 184-210. doi: 10.1214/17-aihp880.
[120]	P. Henry-Labordère, X. Tan and N. Touzi, A numerical algorithm for a class of BSDEs via the branching process, Stochastic Process. Appl., 124 (2014), 1112-1140. doi: 10.1016/j.spa.2013.10.005.
[121]	J. Hermann, Z. Schätzle and F. Noé, Deep-neural-network solution of the electronic Schrödinger equation, Nature Chem., 12 (2020), 891-897. doi: 10.1038/s41557-020-0544-y.
[122]	M. Hochbruck and A. Ostermann, Explicit exponential Runge–Kutta methods for semilinear parabolic problems, SIAM J. Numer. Anal., 43 (2005), 1069-1090. doi: 10.1137/040611434.
[123]	M. Hochbruck and A. Ostermann, Exponential Runge–Kutta methods for parabolic problems, Appl. Numer. Math., 53 (2005), 323–339, Tenth Seminar on Numerical Solution of Differential and Differntial-Algebraic Euqations (NUMDIFF-10). doi: 10.1016/j.apnum.2004.08.005.
[124]	F. Hornung, A. Jentzen and D. Salimova, Space-time deep neural network approximations for high-dimensional partial differential equations, arXiv: 2006.02199, 52 pages.
[125]	M. B. Horowitz, A. Damle and J. W. Burdick, Linear Hamilton Jacobi Bellman equations in high dimensions, in 53rd IEEE Conference on Decision and Control, 2014, 5880–5887. doi: 10.1109/CDC.2014.7040310.
[126]	Y. Hu, D. Nualart and X. Song, Malliavin calculus for backward stochastic differential equations and application to numerical solutions, Ann. Appl. Probab., 21 (2011), 2379-2423. doi: 10.1214/11-AAP762.
[127]	Z. Hu, A. D. Jagtap, G. E. Karniadakis and K. Kawaguchi, When do extended physics-informed neural networks (XPINNs) improve generalization?, SIAM J. Sci. Comput., 44 (2022), A3158–A3182. doi: 10.1137/21M1447039.
[128]	C. Huré, H. Pham and X. Warin, Deep backward schemes for high-dimensional nonlinear PDEs, Math. Comp., 89 (2020), 1547-1579. doi: 10.1090/mcom/3514.
[129]	M. Hutzenthaler, A. Jentzen and T. Kruse, Overcoming the curse of dimensionality in the numerical approximation of parabolic partial differential equations with gradient-dependent nonlinearities, Found. Comput. Math., 62 pp. doi: 10.1007/s10208-021-09514-y.
[130]	M. Hutzenthaler, A. Jentzen, T. Kruse, T. Anh Nguyen and P. von Wurstemberger, Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations, Proc. A., 476 (2020), 20190630. doi: 10.1098/rspa.2019.0630.
[131]	M. Hutzenthaler, A. Jentzen, T. Kruse and T. A. Nguyen, Multilevel Picard approximations for high-dimensional semilinear second-order PDEs with Lipschitz nonlinearities, arXiv: 2009.02484, 37 pages.
[132]	M. Hutzenthaler, A. Jentzen, T. Kruse and T. A. Nguyen, A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations, Partial Differ. Equ. Appl., 1 (2020), 1-34. doi: 10.1007/s42985-019-0006-9.
[133]	M. Hutzenthaler, A. Jentzen and P. von Wurstemberger, Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks, Electron. J. Probab., 25 (2020), 73 pages. doi: 10.1214/20-ejp423.
[134]	M. Hutzenthaler and T. Kruse, Multilevel Picard approximations of high-dimensional semilinear parabolic differential equations with gradient-dependent nonlinearities, SIAM J. Numer. Anal., 58 (2020), 929-961. doi: 10.1137/17M1157015.
[135]	P. Imkeller, G. Dos Reis and J. Zhang, Results on numerics for FBSDE with drivers of quadratic growth, in Contemporary Quantitative Finance, Springer, 2010,159–182. doi: 10.1007/978-3-642-03479-4_9.
[136]	S. Ioffe and C. Szegedy, Batch Normalization: Accelerating deep network training by reducing internal covariate shift, arXiv: 1502.03167, 11 pages.
[137]	A. J. Jacquier and M. Oumgari, Deep curve-dependent PDEs for affine rough volatility, arXiv: 1906.02551, 20 pages.
[138]	A. D. Jagtap and G. E. Karniadakis, Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, Commu. Comput. Phys., 28 (2020), 2002-2041. doi: 10.4208/cicp.OA-2020-0164.
[139]	A. D. Jagtap, K. Kawaguchi and G. Em Karniadakis, Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks, Proc. A., 476 (2020), 20200334. doi: 10.1098/rspa.2020.0334.
[140]	A. D. Jagtap, K. Kawaguchi and G. E. Karniadakis, Adaptive activation functions accelerate convergence in deep and physics-informed neural networks, J. Comput. Phys., 404 (2020), 109136. doi: 10.1016/j.jcp.2019.109136.
[141]	A. D. Jagtap, E. Kharazmi and G. E. Karniadakis, Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems, Comput. Methods Appl. Mech. Engrg., 365 (2020), 113028. doi: 10.1016/j.cma.2020.113028.
[142]	A. Jentzen, Pathwise numerical approximation of SPDEs with additive noise under non-global {L}ipschitz coefficients, Potential Anal., 31 (2009), 375–404, URL https://doi.org/10.1007/s11118-009-9139-3. doi: 10.1007/s11118-009-9139-3.
[143]	A. Jentzen, D. Salimova and T. Welti, A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients, Commun. Math. Sci., 19 (2021), 1167-1205. doi: 10.4310/CMS.2021.v19.n5.a1.
[144]	S. Ji, S. Peng, Y. Peng and X. Zhang, Three algorithms for solving high-dimensional fully coupled FBSDEs through deep learning, IEEE Intell. Syst., 35 (2020), 71-84.
[145]	Y. Jiang and J. Li, Convergence of the deep BSDE method for FBSDEs with non-Lipschitz coefficients, Prob. Uncertain. Quant. Risk, 6 (2021), 391. doi: 10.3934/puqr.2021019.
[146]	L. Jianyu, L. Siwei, Q. Yingjian and H. Yaping, Numerical solution of elliptic partial differential equation using radial basis function neural networks, Neural Netw., 16 (2003), 729-734. doi: 10.1016/S0893-6080(03)00083-2.
[147]	X. Jin, S. Cai, H. Li and G. E. Karniadakis, NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations, J. Comput. Phys., 426 (2021), 109951. doi: 10.1016/j.jcp.2020.109951.
[148]	I. Karatzas and S. Shreve, Brownian Motion and Stochastic Calculus, 2nd edition, Springer New York, NY, 1998. doi: 10.1007/978-1-4612-0949-2.
[149]	G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang and L. Yang, Physics-informed machine learning, Nat. Rev. Phys., 3 (2021), 422-440. doi: 10.1038/s42254-021-00314-5.
[150]	S. Karumuri, R. Tripathy, I. Bilionis and J. Panchal, Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks, J. Comput. Phys., 404 (2020), 109120. doi: 10.1016/j.jcp.2019.109120.
[151]	E. Kharazmi, Z. Zhang and G. E. Karniadakis, Variational physics-informed neural networks for solving partial differential equations, arXiv: 1912.00873, 24 pages.
[152]	E. Kharazmi, Z. Zhang and G. E. Karniadakis, hp-VPINNs: Variational physics-informed neural networks with domain decomposition, Comput. Methods Appl. Mech. Engrg., 374 (2021), 113547. doi: 10.1016/j.cma.2020.113547.
[153]	Y. Khoo, J. Lu and L. Ying, Solving for high-dimensional committor functions using artificial neural networks, Res. Math. Sci., 6. doi: 10.1007/s40687-018-0160-2.
[154]	Y. Khoo, J. Lu and L. Ying, Solving parametric PDE problems with artificial neural networks, European J. Appl. Math., 1–15. doi: 10.1017/S0956792520000182.
[155]	Y. Khoo and L. Ying, SwitchNet: A neural network model for forward and inverse scattering problems, SIAM J. Sci. Comput., 41 (2019), A3182–A3201. doi: 10.1137/18M1222399.
[156]	D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv: 1412.6980, 15 pages.
[157]	G. Kissas, Y. Yang, E. Hwuang, W. R. Witschey, J. A. Detre and P. Perdikaris, Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks, Comput. Methods Appl. Mech. Engrg., 358 (2020), 112623. doi: 10.1016/j.cma.2019.112623.
[158]	S. Kremsner, A. Steinicke and M. Szölgyenyi, A deep neural network algorithm for semilinear elliptic PDEs with applications in insurance mathematics, Risks, 8. doi: 10.3390/risks8040136.
[159]	A. Krishnapriyan, A. Gholami, S. Zhe, R. Kirby and M. W. Mahoney, Characterizing possible failure modes in physics-informed neural networks, in Advances in Neural Information Processing Systems (eds. M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang and J. W. Vaughan), vol. 34, Curran Associates, Inc., 2021, 26548–26560.
[160]	G. Kutyniok, P. Petersen, M. Raslan and R. Schneider, A theoretical analysis of deep neural networks and parametric PDEs, Constr. Approx., 55 (2022), 73-125. doi: 10.1007/s00365-021-09551-4.
[161]	C. Labart and J. Lelong, A parallel algorithm for solving BSDEs, Monte Carlo Methods Appl., 19 (2013), 11-39. doi: 10.1515/mcma-2013-0001.
[162]	I. E. Lagaris, A. Likas and D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9 (1998), 987-1000. doi: 10.1109/72.712178.
[163]	A. Le Cavil, N. Oudjane and F. Russo, Particle system algorithm and chaos propagation related to non-conservative McKean type stochastic differential equations, Stoch. Partial Differ. Equ. Anal. Comput., 5 (2017), 1-37. doi: 10.1007/s40072-016-0079-9.
[164]	A. Le Cavil, N. Oudjane and F. Russo, Monte-Carlo algorithms for a forward Feynman–Kac-type representation for semilinear nonconservative partial differential equations, Monte Carlo Methods Appl., 24 (2018), 55-70. doi: 10.1515/mcma-2018-0005.
[165]	A. Le Cavil, N. Oudjane and F. Russo, Forward Feynman-Kac type representation for semilinear non-conservative partial differential equations, Stochastics, 91 (2019), 1206-1248. doi: 10.1080/17442508.2019.1594809.
[166]	J.-P. Lemor, E. Gobet and X. Warin, Rate of convergence of an empirical regression method for solving generalized backward stochastic differential equations, Bernoulli, 12 (2006), 889-916. doi: 10.3150/bj/1161614951.
[167]	J. Li, J. Chen and B. Li, Gradient-optimized physics-informed neural networks (GOPINNs): a deep learning method for solving the complex modified KdV equation, Nonlinear Dyn., 107 (2022), 781-792. doi: 10.1007/s11071-021-06996-x.
[168]	K. Li, K. Tang, T. Wu and Q. Liao, D3M: A deep domain decomposition method for partial differential equations, IEEE Access, 8 (2020), 5283-5294. doi: 10.1109/ACCESS.2019.2957200.
[169]	Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart and A. Anandkumar, Fourier neural operator for parametric partial differential equations, arXiv: 2010.08895, 16 pages.
[170]	A. T. Lin, S. W. Fung, W. Li, L. Nurbekyan and S. J. Osher, Alternating the population and control neural networks to solve high-dimensional stochastic mean-field games, Proc. Natl. Acad. Sci. USA, 118 (2021), e2024713118. doi: 10.1073/pnas.2024713118.
[171]	Z. Long, Y. Lu and B. Dong, PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network, J. Comput. Phys., 399 (2019), 108925. doi: 10.1016/j.jcp.2019.108925.
[172]	Z. Long, Y. Lu, X. Ma and B. Dong, PDE-Net: Learning PDEs from data, in Proceedings of the 35th International Conference on Machine Learning (eds. J. Dy and A. Krause), vol. 80, PMLR, 2018, 3208–3216.
[173]	M. Longo, S. Mishra, T. K. Rusch and C. Schwab, Higher-order quasi-Monte Carlo training of deep neural networks, SIAM J. Sci. Comput., 43 (2021), A3938–A3966. doi: 10.1137/20M1369373.
[174]	L. Lu, X. Meng, Z. Mao and G. E. Karniadakis, DeepXDE: a deep learning library for solving differential equations, SIAM Rev., 63 (2021), 208-228. doi: 10.1137/19M1274067.
[175]	D. Luo and B. K. Clark, Backflow transformations via neural networks for quantum many-body wave functions, Phys. Rev. Lett., 122 (2019), 226401. doi: 10.1103/PhysRevLett.122.226401.
[176]	T. Luo and H. Yang, Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory, arXiv: 2006.15733, 30 pages.
[177]	K. O. Lye, S. Mishra and D. Ray, Deep learning observables in computational fluid dynamics, J. Comput. Phys., 109339. doi: 10.1016/j.jcp.2020.109339.
[178]	K. O. Lye, S. Mishra, D. Ray and P. Chandrashekar, Iterative surrogate model optimization (ISMO): an active learning algorithm for PDE constrained optimization with deep neural networks, Comput. Methods Appl. Mech. Engrg., 374 (2021), Paper No. 113575, 27. doi: 10.1016/j.cma.2020.113575.
[179]	J. Ma, P. Protter, J. San Martín and S. Torres, Numerical method for backward stochastic differential equations, Ann. Appl. Probab., 12 (2002), 302-316. doi: 10.1214/aoap/1015961165.
[180]	M. Magill, F. Qureshi and H. de Haan, Neural networks trained to solve differential equations learn general representations, in Advances in Neural Information Processing Systems, 2018, 4071–4081.
[181]	Z. Mao, A. D. Jagtap and G. E. Karniadakis, Physics-informed neural networks for high-speed flows, Comput. Methods Appl. Mech. Engrg., 360 (2020), 112789. doi: 10.1016/j.cma.2019.112789.
[182]	M. Martinez, J. San Martín and S. Torres, Numerical method for reflected backward stochastic differential equations, Stoch. Anal. Appl., 29 (2011), 1008-1032. doi: 10.1080/07362994.2011.610162.
[183]	L. McClenny and U. Braga-Neto, Self-adaptive physics-informed neural networks using a soft attention mechanism, arXiv: 2009.04544.
[184]	X. Meng, Z. Li, D. Zhang and G. E. Karniadakis, PPINN: Parareal physics-informed neural network for time-dependent PDEs, Comput. Methods Appl. Mech. Engrg., 370. doi: 10.1016/j.cma.2020.113250.
[185]	C. Michoski, M. Milosavljeviç, T. Oliver and D. R. Hatch, Solving differential equations using deep neural networks, Neurocomputing, 399 (2020), 193-212. doi: 10.1016/j.neucom.2020.02.015.
[186]	S. Mishra and R. Molinaro, Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs, IMA J. Numer. Anal., 42 (2021), 981-1022. doi: 10.1093/imanum/drab032.
[187]	S. Mishra and R. Molinaro, Physics informed neural networks for simulating radiative transfer, J. Quant. Spectrosc. Radiat. Transf., 270 (2021), 107705. doi: 10.1016/j.jqsrt.2021.107705.
[188]	D. Misra, Mish: A self regularized non-monotonic activation function, arXiv: 1908.08681, 14 pages.
[189]	J. Müller and M. Zeinhofer, Deep Ritz revisited, arXiv: 1912.03937, 10 pages.
[190]	M. A. Nabian and H. Meidani, A deep learning solution approach for high-dimensional random differential equations, Probab. Engrg. Mech., 57 (2019), 14-25. doi: 10.1016/j.probengmech.2019.05.001.
[191]	T. Nakamura-Zimmerer, Q. Gong and W. Kang, A causality-free neural network method for high-dimensional Hamilton–Jacobi–Bellman equations, in 2020 American Control Conference (ACC), 2020,787–793. doi: 10.23919/ACC45564.2020.9147270.
[192]	N. Nüsken and L. Richter, Solving high-dimensional Hamilton–Jacobi–Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space, Partial Differ. Equ. Appl., 2 (2021), 48. doi: 10.1007/s42985-021-00102-x.
[193]	B. Øksendal, Stochastic Differential Equations: An Introduction with Applications, 5th edition, Springer Berlin Heidelberg, 1998.
[194]	G. Pang, M. D'Elia, M. Parks and G. Karniadakis, nPINNs: Nonlocal physics-informed neural networks for a parametrized nonlocal universal Laplacian operator. Algorithms and applications, J. Comput. Phys., 422 (2020), 109760. doi: 10.1016/j.jcp.2020.109760.
[195]	G. Pang, L. Lu and G. E. Karniadakis, fPINNs: Fractional physics-informed neural networks, SIAM J. Sci. Comput., 41 (2019), A2603–A2626. doi: 10.1137/18M1229845.
[196]	E. Pardoux and S. Peng, Backward stochastic differential equations and quasilinear parabolic partial differential equations, in Stochastic partial differential equations and their applications (Charlotte, NC, 1991), vol. 176 of Lect. Notes Control Inf. Sci., Springer, Berlin, 1992,200–217. doi: 10.1007/BFb0007334.
[197]	E. Pardoux and S. G. Peng, Adapted solution of a backward stochastic differential equation, Systems Control Lett., 14 (1990), 55-61. doi: 10.1016/0167-6911(90)90082-6.
[198]	S. G. Peng, Probabilistic interpretation for systems of quasilinear parabolic partial differential equations, Stochastics Stochastics Rep., 37 (1991), 61-74. doi: 10.1080/17442509108833727.
[199]	M. A. Pereira, Z. Wang, T. Chen, E. Reed and E. Theodorou, Feynman–Kac Neural Network Architectures for Stochastic Control Using Second-Order FBSDE Theory, in Proceedings of the 2nd Conference on Learning for Dynamics and Control, vol. 120, PMLR, 2020,728–738.
[200]	M. A. Pereira, Z. Wang, I. Exarchos and E. A. Theodorou, Learning Deep Stochastic Optimal Control Policies Using Forward-Backward SDEs, in Robotics: Science and Systems, 2019.
[201]	G. Peskir and A. Shiryaev, Optimal stopping and free-boundary problems, Lectures in Mathematics ETH Zürich, Birkhäuser Verlag, Basel, 2006.
[202]	D. Pfau, J. S. Spencer, A. G. D. G. Matthews and W. M. C. Foulkes, Ab initio solution of the many-electron Schrödinger equation with deep neural networks, Phys. Rev. Research, 2 (2020), 033429. doi: 10.1103/PhysRevResearch.2.033429.
[203]	H. Pham, X. Warin and M. Germain, Neural networks-based backward scheme for fully nonlinear PDEs, Partial Differ. Equ. Appl., 2 (2021), Paper No. 16, 24. doi: 10.1007/s42985-020-00062-8.
[204]	M. Raissi, Deep hidden physics models: Deep learning of nonlinear partial differential equations, J. Mach. Learn. Res., 19 (2018), 932-955.
[205]	M. Raissi, Forward-backward stochastic neural networks: Deep learning of high-dimensional partial differential equations, arXiv: 1804.07010, 17 pages.
[206]	M. Raissi, P. Perdikaris and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378 (2019), 686-707. doi: 10.1016/j.jcp.2018.10.045.
[207]	M. Raissi, P. Perdikaris and G. E. Karniadakis, Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations, arXiv: 1711.10561, 22 pages.
[208]	M. Raissi, P. Perdikaris and G. E. Karniadakis, Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations, arXiv: 1711.10566, 19 pages.
[209]	M. Raissi, P. Perdikaris and G. E. Karniadakis, Numerical Gaussian processes for time-dependent and nonlinear partial differential equations, SIAM J. Sci. Comput., 40 (2018), A172–A198. doi: 10.1137/17M1120762.
[210]	C. Reisinger and Y. Zhang, Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems, Anal. Appl. (Singap.), 18 (2020), 951-999. doi: 10.1142/S0219530520500116.
[211]	A. Richou, Numerical simulation of BSDEs with drivers of quadratic growth, Ann. Appl. Probab., 21 (2011), 1933-1964. doi: 10.1214/10-AAP744.
[212]	A. Richou, Markovian quadratic and superquadratic BSDEs with an unbounded terminal condition, Stochastic Process. Appl., 122 (2012), 3173-3208. doi: 10.1016/j.spa.2012.05.015.
[213]	L. Richter, L. Sallandt and N. Nüsken, Solving high-dimensional parabolic PDEs using the tensor train format, in Proceedings of the 38th International Conference on Machine Learning (eds. M. Meila and T. Zhang), vol. 139 of Proceedings of Machine Learning Research, PMLR, 2021, 8998–9009.
[214]	M. J. Ruijter and C. W. Oosterlee, A Fourier cosine method for an efficient computation of solutions to BSDEs, SIAM J. Sci. Comput., 37 (2015), A859–A889. doi: 10.1137/130913183.
[215]	M. J. Ruijter and C. W. Oosterlee, Numerical Fourier method and second-order Taylor scheme for backward SDEs in finance, Appl. Numer. Math., 103 (2016), 1-26. doi: 10.1016/j.apnum.2015.12.003.
[216]	L. Ruthotto, S. J. Osher, W. Li, L. Nurbekyan and S. W. Fung, A machine learning framework for solving high-dimensional mean field game and mean field control problems, Proc. Natl. Acad. Sci. USA, 117 (2020), 9183-9193. doi: 10.1073/pnas.1922204117.
[217]	M. Sabate Vidales, D. Šiška and L. Szpruch, Unbiased deep solvers for linear parametric PDEs, Appl. Math. Finance, 28 (2021), 299-329. doi: 10.1080/1350486X.2022.2030773.
[218]	E. Samaniego, C. Anitescu, S. Goswami, V. Nguyen-Thanh, H. Guo, K. Hamdia, X. Zhuang and T. Rabczuk, An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications, Comput. Methods Appl. Mech. Engrg., 362 (2020), 112790. doi: 10.1016/j.cma.2019.112790.
[219]	H. Sheng and C. Yang, PFNN: A penalty-free neural network method for solving a class of second-order boundary-value problems on complex geometries, J. Comput. Phys., 428 (2021), 110085. doi: 10.1016/j.jcp.2020.110085.
[220]	Y. Shin, J. Darbon and G. E. Karniadakis, On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs, Comm. Comput. Phys., 28 (2020), 2042-2074. doi: 10.4208/cicp.OA-2020-0193.
[221]	J. Sirignano and K. Spiliopoulos, DGM: A deep learning algorithm for solving partial differential equations, J. Comput. Phys., 375 (2018), 1339-1364. doi: 10.1016/j.jcp.2018.08.029.
[222]	A. V. Skorokhod, Branching diffusion processes, Theory Probab. Its Appl., 9 (1964), 445-449.
[223]	L. Teng, A. Lapitckii and M. Günther, A multi-step scheme based on cubic spline for solving backward stochastic differential equations, Appl. Numer. Math., 150 (2020), 117-138. doi: 10.1016/j.apnum.2019.09.016.
[224]	T. Uchiyama and N. Sonehara, Solving inverse problems in nonlinear PDEs by recurrent neural networks, in IEEE International Conference on Neural Networks, IEEE, 1993, 99–102. doi: 10.1109/ICNN.1993.298524.
[225]	P. van Moerbeke, On optimal stopping and free boundary problems, Arch. Rational Mech. Anal., 60 (1976), 101-148. doi: 10.1007/BF00250676.
[226]	S. Wang and P. Perdikaris, Deep learning of free boundary and Stefan problems, J. Comput. Phys., 109914. doi: 10.1016/j.jcp.2020.109914.
[227]	S. Wang, Y. Teng and P. Perdikaris, Understanding and mitigating gradient flow pathologies in physics-informed neural networks, SIAM J. Sci. Comput., 43 (2021), A3055–A3081. doi: 10.1137/20M1318043.
[228]	S. Wang, H. Wang and P. Perdikaris, On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks, Comput. Methods Appl. Mech. Engrg., 384 (2021), 113938. doi: 10.1016/j.cma.2021.113938.
[229]	Z. Wang and Z. Zhang, A mesh-free method for interface problems using the deep learning approach, J. Comput. Phys., 400 (2020), 108963. doi: 10.1016/j.jcp.2019.108963.
[230]	X. Warin, Variations on branching methods for non linear PDEs, arXiv: 1701.07660, 25 pages.
[231]	X. Warin, Monte Carlofor high-dimensional degenerated semi linear and full non linear PDEs, arXiv: 1805.05078, 23 pages.
[232]	X. Warin, Nesting Monte Carlo for high-dimensional non-linear PDEs, SIAM J. Numer. Anal., 24 (2018), 225-247. doi: 10.1515/mcma-2018-2020.
[233]	S. Watanabe, On the branching process for Brownian particles with an absorbing boundary, J. Math. Kyoto Univ., 4 (1965), 385-398. doi: 10.1215/kjm/1250524667.
[234]	C. L. Wight and J. Zhao, Solving Allen-Cahn and Cahn-Hilliard equations using the adaptive physics informed neural networks, Comm. Comput. Phys., 29 (2021), 930-954. doi: 10.4208/cicp.OA-2020-0086.
[235]	H. Xu, J. Chen and F. Ma, Adaptive deep learning approximation for Allen-Cahn equation, in Computational Science-ICCS 2022: 22nd International Conference, London, UK, June 21-23, 2022, Proceedings, Part IV, Springer-Verlag, Berlin, Heidelberg, 2022,271-283. doi: 10.1007/978-3-031-08760-8_23.
[236]	L. Yang, D. Zhang and G. E. Karniadakis, Physics-informed generative adversarial networks for stochastic differential equations, SIAM J. Sci. Comput., 42 (2020), A292–A317. doi: 10.1137/18M1225409.
[237]	J. Yu, L. Lu, X. Meng and G. E. Karniadakis, Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems, Comput. Methods Appl. Mech. Engrg., 393 (2022), 114823. doi: 10.1016/j.cma.2022.114823.
[238]	L. Yuan, Y.-Q. Ni, X.-Y. Deng and S. Hao, A-PINN: Auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations, J. Comput. Phys., Article No. 111260, Early access version available online. doi: 10.1016/j.jcp.2022.111260.
[239]	Y. Zang, G. Bao, X. Ye and H. Zhou, Weak adversarial networks for high-dimensional partial differential equations, J. Comput. Phys., 411 (2020), 109409. doi: 10.1016/j.jcp.2020.109409.
[240]	D. Zhang, L. Guo and G. E. Karniadakis, Learning in modal space: Solving time-dependent stochastic PDEs using physics-informed neural networks, SIAM J. Sci. Comput., 42 (2020), A639–A665. doi: 10.1137/19M1260141.
[241]	D. Zhang, L. Lu, L. Guo and G. E. Karniadakis, Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems, J. Comput. Phys., 397 (2019), 108850. doi: 10.1016/j.jcp.2019.07.048.
[242]	W. Zhao, Y. Fu and T. Zhou, New kinds of high-order multistep schemes for coupled forward backward stochastic differential equations, SIAM J. Sci. Comput., 36 (2014), A1731–A1751. doi: 10.1137/130941274.
[243]	W. Zhao, G. Zhang and L. Ju, A stable multistep scheme for solving backward stochastic differential equations, SIAM J. Numer. Anal., 48 (2010), 1369-1394. doi: 10.1137/09076979X.
[244]	Y. Zhu, N. Zabaras, P.-S. Koutsourelakis and P. Perdikaris, Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data, J. Comput. Phys., 394 (2019), 56-81. doi: 10.1016/j.jcp.2019.05.024.
[245]	K. Zubov, Z. McCarthy, Y. Ma, F. Calisto, V. Pagliarino, S. Azeglio, L. Bottero, E. Luján, V. Sulzer, A. Bharambe, N. Vinchhi, K. Balakrishnan, D. Upadhyay and C. Rackauckas, NeuralPDE: Automating Physics-Informed Neural Networks (PINNs) with Error Approximations, arXiv: 2107.09443, 77 pages.