# American Institute of Mathematical Sciences

• Previous Article
Comparative study of macroscopic traffic flow models at road junctions
• NHM Home
• This Issue
• Next Article
A new mixed finite element method for the n-dimensional Boussinesq problem with temperature-dependent viscosity
June  2020, 15(2): 247-259. doi: 10.3934/nhm.2020011

## Deep neural network approach to forward-inverse problems

 1 Department of Mathematics, Pohang University of Science and Technology, South Korea 2 Department of Mathematics and Statistics, California State University Long Beach, US

* Corresponding author: Hyung Ju Hwang

Received  January 2020 Revised  April 2020 Published  April 2020

In this paper, we construct approximated solutions of Differential Equations (DEs) using the Deep Neural Network (DNN). Furthermore, we present an architecture that includes the process of finding model parameters through experimental data, the inverse problem. That is, we provide a unified framework of DNN architecture that approximates an analytic solution and its model parameters simultaneously. The architecture consists of a feed forward DNN with non-linear activation functions depending on DEs, automatic differentiation [2], reduction of order, and gradient based optimization method. We also prove theoretically that the proposed DNN solution converges to an analytic solution in a suitable function space for fundamental DEs. Finally, we perform numerical experiments to validate the robustness of our simplistic DNN architecture for 1D transport equation, 2D heat equation, 2D wave equation, and the Lotka-Volterra system.

Citation: Hyeontae Jo, Hwijae Son, Hyung Ju Hwang, Eun Heui Kim. Deep neural network approach to forward-inverse problems. Networks & Heterogeneous Media, 2020, 15 (2) : 247-259. doi: 10.3934/nhm.2020011
##### References:

show all references

##### References:
Network architecture
Experimental result for 1D transport equation
Experimental result for 2D heat equation with $u(0,x,y) = x(1-x)y(1-y)$
Experimental result for 2D heat equation with $u(0,x,y) = 1 \text{, if } (x,y) \in \Omega, 0 \text{, otherwise}$
Experimental result for 2D wave equation
Experimental result for Lotka-Volterra equation
Experimental result for CFL condition
 Algorithm 1: Training 1: procedure train(number of epochs) 2:   Initialize the nerural network.3:   For number of epochs do 4:     sample $z^1, z^2,..., z^m$ from uniform distribution over $\Omega$ 5:     sample $z_I^1, z_I^2,..., z_I^m$ from uniform distribution over $\{0\} \times\Omega$ 6:     sample $z_B^1, z_B^2,..., z_B^m$ from uniform distribution over $\partial\Omega$ 7:     sample k observation points $z_O^1, z_O^2,..., z_O^k$ 8:     Find the true value $u_j = u_p(z_O^j)$ for $j=1,2,...,k$ 9:     Update the neural network by descending its stochastic gradient : $$$\nonumber \nabla_{w, b} [\frac{1}{m} \sum\limits_{i = 1}^m [L_p(u_N)(z^i)^2 + (u_N(z_I^i)-f(z_I^i))^2 + (u_N(z_B^i)-g(z_B^i))^2] + \frac{1}{k}\sum\limits_{j = 1}^k (u_N(z_O^j)-u_j)^2]$$$ 10:   end for 11: end procedure
 Algorithm 1: Training 1: procedure train(number of epochs) 2:   Initialize the nerural network.3:   For number of epochs do 4:     sample $z^1, z^2,..., z^m$ from uniform distribution over $\Omega$ 5:     sample $z_I^1, z_I^2,..., z_I^m$ from uniform distribution over $\{0\} \times\Omega$ 6:     sample $z_B^1, z_B^2,..., z_B^m$ from uniform distribution over $\partial\Omega$ 7:     sample k observation points $z_O^1, z_O^2,..., z_O^k$ 8:     Find the true value $u_j = u_p(z_O^j)$ for $j=1,2,...,k$ 9:     Update the neural network by descending its stochastic gradient : $$$\nonumber \nabla_{w, b} [\frac{1}{m} \sum\limits_{i = 1}^m [L_p(u_N)(z^i)^2 + (u_N(z_I^i)-f(z_I^i))^2 + (u_N(z_B^i)-g(z_B^i))^2] + \frac{1}{k}\sum\limits_{j = 1}^k (u_N(z_O^j)-u_j)^2]$$$ 10:   end for 11: end procedure
Information of grid and observation points
 Data Generation Grid Range Number of Grid Points Number of Observations 1D Transport $(t,x) \in [0,1]\times[0,1]$ $17 \times 100$ 17 2D Heat $(t,x,y) \in [0,1]\times[0,1]\times[0,1]$ $100 \times 100 \times 100$ 13 2D Wave $(t,x,y) \in [0,1]\times[0,1]\times[0,1]$ $100 \times 100 \times 100$ 61 Lotka-Volterra $t \in [0,100]$ 20,000 40
 Data Generation Grid Range Number of Grid Points Number of Observations 1D Transport $(t,x) \in [0,1]\times[0,1]$ $17 \times 100$ 17 2D Heat $(t,x,y) \in [0,1]\times[0,1]\times[0,1]$ $100 \times 100 \times 100$ 13 2D Wave $(t,x,y) \in [0,1]\times[0,1]\times[0,1]$ $100 \times 100 \times 100$ 61 Lotka-Volterra $t \in [0,100]$ 20,000 40
Neural network architecture
 Neural Network Architecture Fully Connected Layers Activation Functions Learning Rate 1D Transport 2(input)-128-256-128-1(output) ReLU $10^{-5}$ 2D Heat 3(input)-128-128-1(output) Sin, Sigmoid $10^{-5}$ 2D Wave 3(input)-128-256-128-1(output) Sin, Tanh $10^{-5}$ Lotka-Volterra 1(input)-64-64-2(output) Sin $10^{-4}$
 Neural Network Architecture Fully Connected Layers Activation Functions Learning Rate 1D Transport 2(input)-128-256-128-1(output) ReLU $10^{-5}$ 2D Heat 3(input)-128-128-1(output) Sin, Sigmoid $10^{-5}$ 2D Wave 3(input)-128-256-128-1(output) Sin, Tanh $10^{-5}$ Lotka-Volterra 1(input)-64-64-2(output) Sin $10^{-4}$
 [1] Zengyun Wang, Jinde Cao, Zuowei Cai, Lihong Huang. Finite-time stability of impulsive differential inclusion: Applications to discontinuous impulsive neural networks. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2677-2692. doi: 10.3934/dcdsb.2020200 [2] Changpin Li, Zhiqiang Li. Asymptotic behaviors of solution to partial differential equation with Caputo–Hadamard derivative and fractional Laplacian: Hyperbolic case. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021023 [3] Xiaochen Mao, Weijie Ding, Xiangyu Zhou, Song Wang, Xingyong Li. Complexity in time-delay networks of multiple interacting neural groups. Electronic Research Archive, , () : -. doi: 10.3934/era.2021022 [4] Alexandr Mikhaylov, Victor Mikhaylov. Dynamic inverse problem for Jacobi matrices. Inverse Problems & Imaging, 2019, 13 (3) : 431-447. doi: 10.3934/ipi.2019021 [5] Armin Lechleiter, Tobias Rienmüller. Factorization method for the inverse Stokes problem. Inverse Problems & Imaging, 2013, 7 (4) : 1271-1293. doi: 10.3934/ipi.2013.7.1271 [6] Zhiming Guo, Zhi-Chun Yang, Xingfu Zou. Existence and uniqueness of positive solution to a non-local differential equation with homogeneous Dirichlet boundary condition---A non-monotone case. Communications on Pure & Applied Analysis, 2012, 11 (5) : 1825-1838. doi: 10.3934/cpaa.2012.11.1825 [7] Sergei Avdonin, Julian Edward. An inverse problem for quantum trees with observations at interior vertices. Networks & Heterogeneous Media, 2021, 16 (2) : 317-339. doi: 10.3934/nhm.2021008 [8] Julian Tugaut. Captivity of the solution to the granular media equation. Kinetic & Related Models, 2021, 14 (2) : 199-209. doi: 10.3934/krm.2021002 [9] Xiaochun Gu, Fang Han, Zhijie Wang, Kaleem Kashif, Wenlian Lu. Enhancement of gamma oscillations in E/I neural networks by increase of difference between external inputs. Electronic Research Archive, , () : -. doi: 10.3934/era.2021035 [10] Quan Hai, Shutang Liu. Mean-square delay-distribution-dependent exponential synchronization of chaotic neural networks with mixed random time-varying delays and restricted disturbances. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3097-3118. doi: 10.3934/dcdsb.2020221 [11] Jianli Xiang, Guozheng Yan. The uniqueness of the inverse elastic wave scattering problem based on the mixed reciprocity relation. Inverse Problems & Imaging, 2021, 15 (3) : 539-554. doi: 10.3934/ipi.2021004 [12] Meng-Xue Chang, Bang-Sheng Han, Xiao-Ming Fan. Global dynamics of the solution for a bistable reaction diffusion equation with nonlocal effect. Electronic Research Archive, , () : -. doi: 10.3934/era.2021024 [13] Mohsen Abdolhosseinzadeh, Mir Mohammad Alipour. Design of experiment for tuning parameters of an ant colony optimization method for the constrained shortest Hamiltonian path problem in the grid networks. Numerical Algebra, Control & Optimization, 2021, 11 (2) : 321-332. doi: 10.3934/naco.2020028 [14] Stefano Bianchini, Paolo Bonicatto. Forward untangling and applications to the uniqueness problem for the continuity equation. Discrete & Continuous Dynamical Systems, 2021, 41 (6) : 2739-2776. doi: 10.3934/dcds.2020384 [15] Andreas Neubauer. On Tikhonov-type regularization with approximated penalty terms. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2021027 [16] Hui Yang, Yuzhu Han. Initial boundary value problem for a strongly damped wave equation with a general nonlinearity. Evolution Equations & Control Theory, 2021  doi: 10.3934/eect.2021019 [17] Emanuela R. S. Coelho, Valéria N. Domingos Cavalcanti, Vinicius A. Peralta. Exponential stability for a transmission problem of a nonlinear viscoelastic wave equation. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021055 [18] Nhu N. Nguyen, George Yin. Stochastic partial differential equation models for spatially dependent predator-prey equations. Discrete & Continuous Dynamical Systems - B, 2020, 25 (1) : 117-139. doi: 10.3934/dcdsb.2019175 [19] Abdulrazzaq T. Abed, Azzam S. Y. Aladool. Applying particle swarm optimization based on Padé approximant to solve ordinary differential equation. Numerical Algebra, Control & Optimization, 2021  doi: 10.3934/naco.2021008 [20] Seddigheh Banihashemi, Hossein Jafaria, Afshin Babaei. A novel collocation approach to solve a nonlinear stochastic differential equation of fractional order involving a constant delay. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021025

2019 Impact Factor: 1.053