# American Institute of Mathematical Sciences

• Previous Article
Finite- and multi-dimensional state representations and some fundamental asymptotic properties of a family of nonlinear multi-population models for HIV/AIDS with ART treatment and distributed delays
• DCDS-S Home
• This Issue
• Next Article
On properties of similarity boundary of attractors in product dynamical systems
doi: 10.3934/dcdss.2021103
Online First

Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Online First articles via the “Online First” tab for the selected journal.

## ISALT: Inference-based schemes adaptive to large time-stepping for locally Lipschitz ergodic systems

 1 Department of Mathematics and Statistics, University of North Carolina at Charlotte, 9201 Univ City Blvd., Charlotte, NC 28023, USA 2 Department of Mathematics, Johns Hopkins University, 3400 N. Charles Street, Baltimore, MD 21218, USA 3 Department of Mathematics and Statistics, State University of New York at Albany, Earth Science 110, 1400 Washington Avenue, Albany, NY 12222, USA

* Corresponding author: Fei Lu

Received  February 2021 Revised  June 2021 Early access September 2021

Fund Project: XL is supported by NSF DMS CAREER-1847770. FL is supported NSF DMS 1913243 and NSF DMS 1821211. FY is supported by AMS-Simons travel grants

Efficient simulation of SDEs is essential in many applications, particularly for ergodic systems that demand efficient simulation of both short-time dynamics and large-time statistics. However, locally Lipschitz SDEs often require special treatments such as implicit schemes with small time-steps to accurately simulate the ergodic measures. We introduce a framework to construct inference-based schemes adaptive to large time-steps (ISALT) from data, achieving a reduction in time by several orders of magnitudes. The key is the statistical learning of an approximation to the infinite-dimensional discrete-time flow map. We explore the use of numerical schemes (such as the Euler-Maruyama, the hybrid RK4, and an implicit scheme) to derive informed basis functions, leading to a parameter inference problem. We introduce a scalable algorithm to estimate the parameters by least squares, and we prove the convergence of the estimators as data size increases.

We test the ISALT on three non-globally Lipschitz SDEs: the 1D double-well potential, a 2D multiscale gradient system, and the 3D stochastic Lorenz equation with a degenerate noise. Numerical results show that ISALT can tolerate time-step magnitudes larger than plain numerical schemes. It reaches optimal accuracy in reproducing the invariant measure when the time-step is medium-large.

Citation: Xingjie Helen Li, Fei Lu, Felix X.-F. Ye. ISALT: Inference-based schemes adaptive to large time-stepping for locally Lipschitz ergodic systems. Discrete & Continuous Dynamical Systems - S, doi: 10.3934/dcdss.2021103
##### References:

show all references

##### References:
Schematic plot of inferring explicit scheme with a large time-step
Large-time statistics for 1D double-well potential. (a) TVD between the empirical invariant densities (PDF) of the inferred schemes and the reference PDF from data. (b) and (c): PDFs and ACFs comparison between the IS-RK4 with $c_0$ excluded and the reference data
1D double-well potential: Convergence of estimators in IS-RK4 with $c_0$ excluded. (a) The relative error of the estimator $\widehat{c_1^{{\delta}, N,M}}$ with ${\delta} = 80\times \Delta t$ converges at an order about $(MN)^{-1/2}$, matching Theorem 3.5. (b) Left column: The coefficients depend on the time-step ${\delta} = {\mathrm{Gap}}\times \Delta t$, with $c_1$ being almost 1 and $c_2$ being close to linear in ${\delta}$ until ${\delta}>0.08$. The error bars, which are too narrow to be seen, are the standard deviations of the single-trajectory estimators from the $M$-trajectory estimator. Right column: The residual decays at an order $O({\delta}^{1/2})$, matching Theorem 3.6
Large-time statistics for the 2D gradient system. (a) TVD between the $x_1$ marginal invariant densities (PDF) of the inferred schemes and the reference PDF from data. (b) and (c): PDFs and ACFs comparison between IS-SSBE with $c_0$ excluded and the reference data
2D gradient system: Convergence of estimators in IS-SSBE with $c_0$ excluded. (a) The relative error of the estimator $\widehat{c_1^{{\delta}, N,M}}$ with ${\delta} = 120 \Delta t$ converges at an order about $(MN)^{-1/2}$, matching Theorem 3.5. (b) Left column: The estimators of $c_1, c_2$ are almost linear in ${\delta}$. Right column: The residual changes little as ${\delta}$ decreases, due to that IS-SSBE is not a parametrization of an explicit scheme (thus, Theorem 3.6 does not apply)
2D gradient system: Convergence of estimators in IS-RK4 with $c_0$ excluded. (a) The relative error of the estimator $\widehat{c_1^{{\delta}, N,M}}$ with ${\delta} = 120 \Delta t$ converges at an order about $(MN)^{-1/2}$, matching Theorem 3.5. (b) Left column: The estimators of $c_1, c_2$ are constant for all ${\delta}$. Right column: The residual decays at an order $O({\delta}^{1/2})$, matching Theorem 3.6
Large-time statistics of $x_1$ for the stochastic Lorenz system. (a) TVD between the $x_1$ marginal invariant densities (PDF) of the inferred schemes and the reference PDF from data. (b) and (c): PDFs and ACFs comparison between IS-RK4 with $c_0$ included and the reference data
ACF and PDF of $x_3$ in the stochastic Lorenz system. Similar to the other examples, IS-RK4 (with $c_0$ included) reproduces the PDF and the ACF the best when the time-step is medium large, while plain RK4 and IS-EM blow up even when ${\mathrm{Gap}} = 20$
The 3D stochastic Lorenz system: Convergence of estimators in IS-RK4 with $c_0$ included. (a) The relative error of the estimator $\widehat{c_1^{{\delta}, N,M}}$ with ${\delta} = 240 \Delta t = 0.12$ converges at order about $(MN)^{-1/2}$, matching Theorem 3.5. (b) Left column: The estimators of $c_0,c_1, c_2$ are varies little until ${\delta}>0.12$. The vertical dash line is the optimal time gap. Right column: The residuals decay at orders slightly higher than $O({\delta}^{1/2})$
Notations
 Notation Description ${{\bf X}}_t$ and ${{\bf B}}_t$ true state process and original stochastic force $f({{\bf X}}_t)$, $\sigma\in \mathbb{R}^{d\times m}$ local-Lipschitz drift and diffusion matrix $dt$ time-step generating data ${\delta}= {\mathrm{Gap}} \times dt$ time-step for inferred scheme, ${\mathrm{Gap}}\in \{ 1, 2, 4, 10, 20, 40,\ldots\}$ $t_i = i{\delta}$ discrete time instants of data $\{{{\bf X}}_{t_0:t_N}^{(m)}, {{\bf B}}_{t_0:t_N}^{(m)}\}_{m=1}^M$ Data: $M$ independent paths of ${{\bf X}}$ and ${{\bf B}}$ at discrete-times $\mathcal{F}\left({{\bf X}}_{t_i},\, {{\bf B}}_{[t_{i}, \, t_{i+1})}\right)$ true flow map representing $({{\bf X}}_{t_{i+1}}-{{\bf X}}_{t_i})/{\delta}$ ${F}^{\delta}({{\bf X}}_{t_n},\Delta {{\bf B}}_{t_n})$ approximate flow map using only ${{\bf X}}_{t_n}$, $\Delta {{\bf B}}_{t_n} = {{\bf B}}_{t_{n+1}}-{{\bf B}}_{t_{n}}$ $\widetilde F^{\delta}\left(c^{\delta}, {{\bf X}}_{t_n},\Delta {{\bf B}}_{t_n} \right)$ parametric approximate flow map $c^{\delta}=(c_0^{\delta},\dots,c_p^{\delta})$ parameters to be estimated for the inferred scheme $\eta_n$ and $\sigma_{\eta}^{\delta}$ iid $N(0, I_d)$ and covariance, representing regression residual EM and IS-EM Euler-Maruyama and inferred scheme (IS) parametrizing it HRK4 and IS-RK4 hybrid RK4 and inferred scheme parametrizing RK4 SSBE and IS-SSBE split-step stochastic backward Euler and IS parametrizing it
 Notation Description ${{\bf X}}_t$ and ${{\bf B}}_t$ true state process and original stochastic force $f({{\bf X}}_t)$, $\sigma\in \mathbb{R}^{d\times m}$ local-Lipschitz drift and diffusion matrix $dt$ time-step generating data ${\delta}= {\mathrm{Gap}} \times dt$ time-step for inferred scheme, ${\mathrm{Gap}}\in \{ 1, 2, 4, 10, 20, 40,\ldots\}$ $t_i = i{\delta}$ discrete time instants of data $\{{{\bf X}}_{t_0:t_N}^{(m)}, {{\bf B}}_{t_0:t_N}^{(m)}\}_{m=1}^M$ Data: $M$ independent paths of ${{\bf X}}$ and ${{\bf B}}$ at discrete-times $\mathcal{F}\left({{\bf X}}_{t_i},\, {{\bf B}}_{[t_{i}, \, t_{i+1})}\right)$ true flow map representing $({{\bf X}}_{t_{i+1}}-{{\bf X}}_{t_i})/{\delta}$ ${F}^{\delta}({{\bf X}}_{t_n},\Delta {{\bf B}}_{t_n})$ approximate flow map using only ${{\bf X}}_{t_n}$, $\Delta {{\bf B}}_{t_n} = {{\bf B}}_{t_{n+1}}-{{\bf B}}_{t_{n}}$ $\widetilde F^{\delta}\left(c^{\delta}, {{\bf X}}_{t_n},\Delta {{\bf B}}_{t_n} \right)$ parametric approximate flow map $c^{\delta}=(c_0^{\delta},\dots,c_p^{\delta})$ parameters to be estimated for the inferred scheme $\eta_n$ and $\sigma_{\eta}^{\delta}$ iid $N(0, I_d)$ and covariance, representing regression residual EM and IS-EM Euler-Maruyama and inferred scheme (IS) parametrizing it HRK4 and IS-RK4 hybrid RK4 and inferred scheme parametrizing RK4 SSBE and IS-SSBE split-step stochastic backward Euler and IS parametrizing it
Inference-based schemes adaptive to large time-stepping (ISALT): detailed algorithm
 Input: Full model; a high fidelity solver preserving the invariant measure. Output: Estimated parametric scheme 1: Generate data: solve the system with the high fidelity solver, which has a small time-step $dt$; down sample to get time series with ${\delta}= \mathrm{Gap}\times dt$. Denote the data, consisting of $M$ independent trajectories on $[0,N{\delta}]$, by $\{{{\bf X}}_{t_0:t_N}^{(m)}, {{\bf B}}_{t_0:t_N}^{(m)}\}_{m=1}^M$ with $t_i= i{\delta}$. 2: Pick a parametric form approximating the flow map (2.1) as in (2.5)–(2.6). 3: Estimate parameters $c_{0:p}^{\delta}$ and $\sigma_\eta$ as in (2.7). 4: Model selection: run the inferred scheme for cross-validation, and test the consistency of the estimators.
 Input: Full model; a high fidelity solver preserving the invariant measure. Output: Estimated parametric scheme 1: Generate data: solve the system with the high fidelity solver, which has a small time-step $dt$; down sample to get time series with ${\delta}= \mathrm{Gap}\times dt$. Denote the data, consisting of $M$ independent trajectories on $[0,N{\delta}]$, by $\{{{\bf X}}_{t_0:t_N}^{(m)}, {{\bf B}}_{t_0:t_N}^{(m)}\}_{m=1}^M$ with $t_i= i{\delta}$. 2: Pick a parametric form approximating the flow map (2.1) as in (2.5)–(2.6). 3: Estimate parameters $c_{0:p}^{\delta}$ and $\sigma_\eta$ as in (2.7). 4: Model selection: run the inferred scheme for cross-validation, and test the consistency of the estimators.
Time gap of blow-up for each scheme: plain verse inferred
 1D double-well 2D gradient system 3D Lorenz system Plain RK4 ${\mathrm{Gap}}=20$ ${\mathrm{Gap}}=20$ ${\mathrm{Gap}}=10$ IS-RK4 ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>400$ Plain SSBE ${\mathrm{Gap}}=40$ ${\mathrm{Gap}}=40$ ${\mathrm{Gap}}=20$ IS-SSBE ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>400$
 1D double-well 2D gradient system 3D Lorenz system Plain RK4 ${\mathrm{Gap}}=20$ ${\mathrm{Gap}}=20$ ${\mathrm{Gap}}=10$ IS-RK4 ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>400$ Plain SSBE ${\mathrm{Gap}}=40$ ${\mathrm{Gap}}=40$ ${\mathrm{Gap}}=20$ IS-SSBE ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>200$ ${\mathrm{Gap}}>400$
 [1] Meixin Xiong, Liuhong Chen, Ju Ming, Jaemin Shin. Accelerating the Bayesian inference of inverse problems by using data-driven compressive sensing method based on proper orthogonal decomposition. Electronic Research Archive, , () : -. doi: 10.3934/era.2021044 [2] Matthew O. Williams, Clarence W. Rowley, Ioannis G. Kevrekidis. A kernel-based method for data-driven koopman spectral analysis. Journal of Computational Dynamics, 2015, 2 (2) : 247-265. doi: 10.3934/jcd.2015005 [3] Michael Herty, Adrian Fazekas, Giuseppe Visconti. A two-dimensional data-driven model for traffic flow on highways. Networks & Heterogeneous Media, 2018, 13 (2) : 217-240. doi: 10.3934/nhm.2018010 [4] Hailiang Liu, Xuping Tian. Data-driven optimal control of a seir model for COVID-19. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021093 [5] Mostafa Karimi, Noor Akma Ibrahim, Mohd Rizam Abu Bakar, Jayanthi Arasan. Rank-based inference for the accelerated failure time model in the presence of interval censored data. Numerical Algebra, Control & Optimization, 2017, 7 (1) : 107-112. doi: 10.3934/naco.2017007 [6] Stefano Almi, Massimo Fornasier, Richard Huber. Data-driven evolutions of critical points. Foundations of Data Science, 2020, 2 (3) : 207-255. doi: 10.3934/fods.2020011 [7] Lianbing She, Nan Liu, Xin Li, Renhai Wang. Three types of weak pullback attractors for lattice pseudo-parabolic equations driven by locally Lipschitz noise. Electronic Research Archive, , () : -. doi: 10.3934/era.2021028 [8] Tieliang Gong, Chen Xu, Hong Chen. Modal additive models with data-driven structure identification. Mathematical Foundations of Computing, 2020, 3 (3) : 165-183. doi: 10.3934/mfc.2020016 [9] Deepak Kumar, Ahmad Jazlan, Victor Sreeram, Roberto Togneri. Partial fraction expansion based frequency weighted model reduction for discrete-time systems. Numerical Algebra, Control & Optimization, 2016, 6 (3) : 329-337. doi: 10.3934/naco.2016015 [10] Mohammad-Sahadet Hossain. Projection-based model reduction for time-varying descriptor systems: New results. Numerical Algebra, Control & Optimization, 2016, 6 (1) : 73-90. doi: 10.3934/naco.2016.6.73 [11] Guangjun Shen, Xueying Wu, Xiuwei Yin. Stabilization of stochastic differential equations driven by G-Lévy process with discrete-time feedback control. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 755-774. doi: 10.3934/dcdsb.2020133 [12] Arnulf Jentzen, Benno Kuckuck, Thomas Müller-Gronbach, Larisa Yaroslavtseva. Counterexamples to local Lipschitz and local Hölder continuity with respect to the initial values for additive noise driven stochastic differential equations with smooth drift coefficient functions with at most polynomially growing derivatives. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021203 [13] María J. Garrido–Atienza, Kening Lu, Björn Schmalfuss. Random dynamical systems for stochastic partial differential equations driven by a fractional Brownian motion. Discrete & Continuous Dynamical Systems - B, 2010, 14 (2) : 473-493. doi: 10.3934/dcdsb.2010.14.473 [14] Sarah Jane Hamilton, Andreas Hauptmann, Samuli Siltanen. A data-driven edge-preserving D-bar method for electrical impedance tomography. Inverse Problems & Imaging, 2014, 8 (4) : 1053-1072. doi: 10.3934/ipi.2014.8.1053 [15] Ying Kou, Zhong Wan. A new data-driven robust optimization approach to multi-item newsboy problems. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021180 [16] Xianming Liu, Guangyue Han. A Wong-Zakai approximation of stochastic differential equations driven by a general semimartingale. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2499-2508. doi: 10.3934/dcdsb.2020192 [17] Bahareh Akhtari, Esmail Babolian, Andreas Neuenkirch. An Euler scheme for stochastic delay differential equations on unbounded domains: Pathwise convergence. Discrete & Continuous Dynamical Systems - B, 2015, 20 (1) : 23-38. doi: 10.3934/dcdsb.2015.20.23 [18] Yanqing Wang. A semidiscrete Galerkin scheme for backward stochastic parabolic differential equations. Mathematical Control & Related Fields, 2016, 6 (3) : 489-515. doi: 10.3934/mcrf.2016013 [19] Weidong Zhao, Jinlei Wang, Shige Peng. Error estimates of the $\theta$-scheme for backward stochastic differential equations. Discrete & Continuous Dynamical Systems - B, 2009, 12 (4) : 905-924. doi: 10.3934/dcdsb.2009.12.905 [20] Weidong Zhao, Yang Li, Guannan Zhang. A generalized $\theta$-scheme for solving backward stochastic differential equations. Discrete & Continuous Dynamical Systems - B, 2012, 17 (5) : 1585-1603. doi: 10.3934/dcdsb.2012.17.1585

2020 Impact Factor: 2.425