# American Institute of Mathematical Sciences

March  2020, 2(1): 55-80. doi: 10.3934/fods.2020004

## Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization

 1 CEREA, joint laboratory École des Ponts ParisTech and EDF R & D, Université Paris-Est, Champs-sur-Marne, France 2 Nansen Environmental and Remote Sensing Center, Bergen, Norway, and Sorbonne University, CNRS-IRD-MNHN, LOCEAN, Paris, France 3 Departement of Meteorology, University of Reading and NCEO, United-Kingdom, and Mathematical Institute, Utrecht University, The Netherlands 4 Nansen Environmental and Remote Sensing Center, Bergen, Norway

* Corresponding author: Marc Bocquet

Published  March 2020

The reconstruction from observations of high-dimensional chaotic dynamics such as geophysical flows is hampered by (ⅰ) the partial and noisy observations that can realistically be obtained, (ⅱ) the need to learn from long time series of data, and (ⅲ) the unstable nature of the dynamics. To achieve such inference from the observations over long time series, it has been suggested to combine data assimilation and machine learning in several ways. We show how to unify these approaches from a Bayesian perspective using expectation-maximization and coordinate descents. In doing so, the model, the state trajectory and model error statistics are estimated all together. Implementations and approximations of these methods are discussed. Finally, we numerically and successfully test the approach on two relevant low-order chaotic models with distinct identifiability.

Citation: Marc Bocquet, Julien Brajard, Alberto Carrassi, Laurent Bertino. Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization. Foundations of Data Science, 2020, 2 (1) : 55-80. doi: 10.3934/fods.2020004
##### References:

show all references

##### References:
From top to bottom: representation of the flow rate $\boldsymbol \phi_ \mathbf{A}$ with a NN, integration of the flow rate into $\mathbf{f}_ \mathbf{A}$ using an explicit integration scheme (here a second-order Runge Kutta scheme), and ${N_\mathrm{c}}-$fold composition up to the full resolvent $\mathbf{F}_ \mathbf{A}$. $\delta t$ is the integration time step corresponding to the resolvent $\mathbf{f}_ \mathbf{A}$
On the left hand side: Properties of the surrogate model obtained from full but noisy observation of the L96 model in the nominal configuration ($L = 4$, $K = 5000$, $\sigma_y = 1$, ${N_\mathrm{y}} = {N_\mathrm{x}} = 40$). On the right hand side: Properties of the surrogate model obtained from full but noisy observation of the L05Ⅲ model in the nominal configuration ($L = 4$, $K = 5000$, $\sigma_y = 1$, ${N_\mathrm{y}} = {N_\mathrm{x}} = 36$). From top to bottom, are plotted the FS (NRMSE as a function of lead time in Lyapunov time), the LS (all exponents), and the PSD (in log-log-scale). A total of $10$ experiments have been performed for both configurations. The curves corresponding to each member are drawn with thin blue lines while the mean of each indicator over the ensemble are drawn in thick dashed orange line
Same as Figure 2 but for several values of the training window length $K$. Each curve is the mean over $10$ experiments with different sets of observations. The LS and PSD of the reference models are also plotted for comparison
On the left hand side: Properties of the surrogate model obtained from full but noisy observation of the L96 model in the nominal configuration ($L = 4$, $K = 5000$, ${N_\mathrm{y}} = {N_\mathrm{x}} = 40$ and with several $\sigma_y$). On the right hand side: Properties of the surrogate model obtained from full but noisy observation of the L05Ⅲ model in the nominal configuration ($L = 4$, $K = 5000$, $\sigma_y = 1$, ${N_\mathrm{y}} = {N_\mathrm{x}} = 36$ and with several $\sigma_y$). From top to bottom, are plotted the FS (NRMSE as a function of lead time in Lyapunov time) and the PSD (in log-log-scale), averaged over an ensemble of $10$ samples
On the left hand side: Properties of the surrogate model obtained from partial and noisy observation of the L96 model in the nominal configuration ($L = 4$, $K = 5000$, $\sigma_y = 1$, ${N_\mathrm{x}} = 40$) where ${N_\mathrm{y}}$ is varied. On the right hand side: Properties of the surrogate model obtained from partial and noisy observation of the L05Ⅲ model in the nominal configuration ($L = 4$, $K = 5000$, $\sigma_y = 1$, ${N_\mathrm{x}} = 36$) where ${N_\mathrm{y}}$ is varied. From top to bottom, are plotted the mean FS (NRMSE as a function of lead time in Lyapunov time), the mean LS (all exponents), and the mean PSD (in log-log-scale). A total of $10$ experiments have been performed for both configurations
Scalar indicators for nominal experiments based on L96 and L05Ⅲ. Key hyperparameters are recalled. The statistics of the indicators are obtained over $10$ samples
 Model ${N_\mathrm{y}}$ $\sigma_y$ $K$ $L$ $\pi_ \frac{1}{2}$ $\sigma_q$ $\lambda_1$ L96 $40$ $1$ $5000$ $4$ $4.56 \pm 0.06$ $0.08790 \pm 2\, 10^{-5}$ $1.66 \pm 0.02$ L05Ⅲ $36$ $1$ $5000$ $4$ $4.06 \pm 0.21$ $0.07720 \pm 2\, 10^{-5}$ $1.03 \pm 0.05$
 Model ${N_\mathrm{y}}$ $\sigma_y$ $K$ $L$ $\pi_ \frac{1}{2}$ $\sigma_q$ $\lambda_1$ L96 $40$ $1$ $5000$ $4$ $4.56 \pm 0.06$ $0.08790 \pm 2\, 10^{-5}$ $1.66 \pm 0.02$ L05Ⅲ $36$ $1$ $5000$ $4$ $4.06 \pm 0.21$ $0.07720 \pm 2\, 10^{-5}$ $1.03 \pm 0.05$
Scalar indicators for L96 and L05Ⅲ in their nominal configuration, using either the full or the approximate schemes. The statistics of the indicators are obtained over $10$ samples
 Model Scheme $\pi_ \frac{1}{2}$ $\sigma_q$ $\lambda_1$ L96 Approximate $4.56 \pm 0.06$ $0.08790 \pm 2\, 10^{-5}$ $1.66 \pm 0.02$ L96 Full $4.24 \pm 0.07$ $0.09152$ $1.66 \pm 0.02$ L05Ⅲ Approximate $4.06 \pm 0.21$ $0.07720 \pm 2\, 10^{-5}$ $1.03 \pm 0.05$ L05Ⅲ Full $3.97 \pm 0.17$ $0.08024$ $1.03 \pm 0.04$
 Model Scheme $\pi_ \frac{1}{2}$ $\sigma_q$ $\lambda_1$ L96 Approximate $4.56 \pm 0.06$ $0.08790 \pm 2\, 10^{-5}$ $1.66 \pm 0.02$ L96 Full $4.24 \pm 0.07$ $0.09152$ $1.66 \pm 0.02$ L05Ⅲ Approximate $4.06 \pm 0.21$ $0.07720 \pm 2\, 10^{-5}$ $1.03 \pm 0.05$ L05Ⅲ Full $3.97 \pm 0.17$ $0.08024$ $1.03 \pm 0.04$
 [1] Kengo Nakai, Yoshitaka Saiki. Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1079-1092. doi: 10.3934/dcdss.2020352 [2] Lars Grüne. Computing Lyapunov functions using deep neural networks. Journal of Computational Dynamics, 2020  doi: 10.3934/jcd.2021006 [3] Leslaw Skrzypek, Yuncheng You. Feedback synchronization of FHN cellular neural networks. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2021001 [4] Mauricio Achigar. Extensions of expansive dynamical systems. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020399 [5] Geir Evensen, Javier Amezcua, Marc Bocquet, Alberto Carrassi, Alban Farchi, Alison Fowler, Pieter L. Houtekamer, Christopher K. Jones, Rafael J. de Moraes, Manuel Pulido, Christian Sampson, Femke C. Vossepoel. An international initiative of predicting the SARS-CoV-2 pandemic using ensemble data assimilation. Foundations of Data Science, 2020  doi: 10.3934/fods.2021001 [6] The Editors. The 2019 Michael Brin Prize in Dynamical Systems. Journal of Modern Dynamics, 2020, 16: 349-350. doi: 10.3934/jmd.2020013 [7] Nitha Niralda P C, Sunil Mathew. On properties of similarity boundary of attractors in product dynamical systems. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021004 [8] Ziang Long, Penghang Yin, Jack Xin. Global convergence and geometric characterization of slow to fast weight evolution in neural network training for classifying linearly non-separable data. Inverse Problems & Imaging, 2021, 15 (1) : 41-62. doi: 10.3934/ipi.2020077 [9] Toshiko Ogiwara, Danielle Hilhorst, Hiroshi Matano. Convergence and structure theorems for order-preserving dynamical systems with mass conservation. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3883-3907. doi: 10.3934/dcds.2020129 [10] Peter Giesl, Zachary Langhorne, Carlos Argáez, Sigurdur Hafstein. Computing complete Lyapunov functions for discrete-time dynamical systems. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 299-336. doi: 10.3934/dcdsb.2020331 [11] Alessandro Fonda, Rodica Toader. A dynamical approach to lower and upper solutions for planar systems "To the memory of Massimo Tarallo". Discrete & Continuous Dynamical Systems - A, 2021  doi: 10.3934/dcds.2021012 [12] Stefan Siegmund, Petr Stehlík. Time scale-induced asynchronous discrete dynamical systems. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 1011-1029. doi: 10.3934/dcdsb.2020151 [13] Guillaume Cantin, M. A. Aziz-Alaoui. Dimension estimate of attractors for complex networks of reaction-diffusion systems applied to an ecological model. Communications on Pure & Applied Analysis, 2021, 20 (2) : 623-650. doi: 10.3934/cpaa.2020283 [14] Gernot Holler, Karl Kunisch. Learning nonlocal regularization operators. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021003 [15] Max E. Gilmore, Chris Guiver, Hartmut Logemann. Sampled-data integral control of multivariable linear infinite-dimensional systems with input nonlinearities. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021001 [16] Jesús A. Álvarez López, Ramón Barral Lijó, John Hunton, Hiraku Nozawa, John R. Parker. Chaotic Delone sets. Discrete & Continuous Dynamical Systems - A, 2021  doi: 10.3934/dcds.2021016 [17] Xin Guo, Lexin Li, Qiang Wu. Modeling interactive components by coordinate kernel polynomial models. Mathematical Foundations of Computing, 2020, 3 (4) : 263-277. doi: 10.3934/mfc.2020010 [18] Min Ji, Xinna Ye, Fangyao Qian, T.C.E. Cheng, Yiwei Jiang. Parallel-machine scheduling in shared manufacturing. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020174 [19] Xiaoxian Tang, Jie Wang. Bistability of sequestration networks. Discrete & Continuous Dynamical Systems - B, 2021, 26 (3) : 1337-1357. doi: 10.3934/dcdsb.2020165 [20] Marcos C. Mota, Regilene D. S. Oliveira. Dynamic aspects of Sprott BC chaotic system. Discrete & Continuous Dynamical Systems - B, 2021, 26 (3) : 1653-1673. doi: 10.3934/dcdsb.2020177

Impact Factor: