\`x^2+y_1+z_12^34\`
Advanced Search
Article Contents
Article Contents

Discrete gradients for computational Bayesian inference

This research has been partially funded by Deutsche Forschungsgemeinschaft (DFG) through grant CRC 1294 'Data Assimilation' (projects A02 and B04)

Abstract Full Text(HTML) Figure(4) / Table(1) Related Papers Cited by
  • In this paper, we exploit the gradient flow structure of continuous-time formulations of Bayesian inference in terms of their numerical time-stepping. We focus on two particular examples, namely, the continuous-time ensemble Kalman–Bucy filter and a particle discretisation of the Fokker–Planck equation associated to Brownian dynamics. Both formulations can lead to stiff differential equations which require special numerical methods for their efficient numerical implementation. We compare discrete gradient methods to alternative semi-implicit and other iterative implementations of the underlying Bayesian inference problems.

    Mathematics Subject Classification: Primary: 62F15, 65C05, 82C22, 65P99; Secondary: 65C35.

    Citation:

    \begin{equation} \\ \end{equation}
  • 加载中
  • Figure 1.  Exact solution and numerical approximations to the mean $ m_\tau $ and variance $ \sigma_\tau $ estimated by the EnKBF using the discrete gradient (DG) method with $ \theta = 1 $ and the semi-implicit (SI) Euler method with step-sizes (a) $ \Delta \tau = 0.1 $, (b) $ \Delta \tau = 0.2 $, (c) $ \Delta \tau = 0.5 $, (d) $ \Delta \tau = 1.0 $

    Figure 2.  Exact final values and numerical approximations to the mean $ m_\tau $ and variance $ \sigma_\tau $ estimated by the EnKBF using the discrete gradient (DG) with $ \theta = 1 $, the semi-implicit (SI) Euler, and the gradient-free, explicit (IEnKF) method with step-sizes (a) $ \Delta \tau = 0.01 $, (b) $ \Delta \tau = 0.1 $, (c) $ \Delta \tau = 0.2 $, (d) $ \Delta \tau = 0.5 $. The reference time evolution of the mean and variance is given by the explicit Euler method with step-size $ \Delta \tau = 0.00025 $

    Figure 3.  Exact final values and numerical approximations to the mean $ m_t $ and variance $ \sigma_t $ for the linear problem. Posterior is estimated by the Fokker-Planck dynamics with $ M = 10 $ particles using the discrete gradient (DG) method with $ \theta = 1 $ and the semi-implicit (SI) Euler method with step-sizes (a) $ \Delta \tau = 0.004 $, (b) $ \Delta \tau = 0.01 $, (c) $ \Delta \tau = 0.04 $, (d) $ \Delta \tau = 0.1 $. The reference time evolution of the mean and variance is given by the Explicit Euler (EE) method with step-size $ \Delta \tau = 2 \times 10^{-4} $

    Figure 4.  Exact final values and numerical approximations to the mean $ m_t $ and variance $ \sigma_t $ for the non-linear problem. Posterior is estimated by the Fokker-Planck dynamics using the discrete gradient (DG) method with $ \theta = 1 $ and the semi-implicit (SI) Euler method with step-sizes (a) $ \Delta \tau = 0.002 $, (b) $ \Delta \tau = 0.005 $, (c) $ \Delta \tau = 0.02 $, (d) $ \Delta \tau = 0.05 $. The reference time evolution of the mean and variance is given by the Explicit Euler (EE) method with step-size $ \Delta \tau = 2.5 \times 10^{-6} $

    Table 1.  RMSE for sequential data assimilation applied to the Lorenz-63 model. The parameter $ \alpha = 1 $ corresponds to a standard square root ensemble Kalman filter, while $ \alpha<1 $ results in a Gaussian mixture approximation to the prior and posterior distributions. Small improvements over the ensemble Kaman filter can be found for ensemble sizes $ M\ge 35 $

    $ \alpha\backslash M $ $ 15 $ $ 20 $ $ 25 $ $ 30 $ $ 35 $ $ 40 $ $ 45 $ $ 50 $
    $ 0.8 $ $ 2.5145 $ $ 2.4726 $ $ 2.4471 $ $ 2.4406 $ $ 2.4221 $ $ 2.4173 $ $ 2.4162 $ $ 2.4112 $
    $ 0.85 $ $ 2.5048 $ $ 2.4585 $ $ 2.4357 $ $ 2.4330 $ $ 2.4186 $ $ 2.4161 $ $ 2.4020 $ $ 2.4026 $
    $ 0.9 $ $ 2.4989 $ $ 2.4493 $ $ 2.4396 $ $ 2.4259 $ $ 2.4177 $ $ 2.4103 $ $ 2.4106 $ $ 2.4075 $
    $ 0.95 $ $ 2.4998 $ $ 2.4605 $ $ 2.4399 $ $ 2.4327 $ $ 2.4242 $ $ 2.4132 $ $ 2.4162 $ $ 2.4131 $
    $ 1.0 $ $ 2.4804 $ $ 2.4597 $ $ 2.4518 $ $ 2.4354 $ $ 2.4334 $ $ 2.4247 $ $ 2.4244 $ $ 2.4203 $
     | Show Table
    DownLoad: CSV
  • [1] J. AmezcuaE. KalnayK. Ide and S. Reich, Ensemble transform Kalman-Bucy filters, Q. J. R. Meteor. Soc., 140 (2014), 995-1004.  doi: 10.1002/qj.2186.
    [2] U. M. Ascher, Numerical Methods for Evolutionary Differential Equations, Computational Science & Engineering, 5. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2008. doi: 10.1137/1.9780898718911.
    [3] K. Bergemann and S. Reich, A localization technique for ensemble Kalman filters, Q. J. R. Meteorological Soc., 136 (2010), 701-707.  doi: 10.1002/qj.591.
    [4] K. Bergemann and S. Reich, A mollified ensemble Kalman filter, Q. J. R. Meteorological Soc., 136 (2010), 1636-1643.  doi: 10.1002/qj.672.
    [5] K. Bergemann and S. Reich, An ensemble Kalman-Bucy filter for continuous data assimilation, Meteorolog. Zeitschrift, 21 (2012), 213-219.  doi: 10.1127/0941-2948/2012/0307.
    [6] D. BlömkerC. Schillings and P. Wacker, A strongly convergent numerical scheme for ensemble Kalman inversion, SIAM J. Numer. Anal., 56 (2018), 2537-2562.  doi: 10.1137/17M1132367.
    [7] Y. Chen and D. S. Oliver, Levenberg-Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification, Computational Geoscience, 17 (2013), 689-703.  doi: 10.1007/s10596-013-9351-5.
    [8] N. ChustagulpromS. Reich and M. Reinhardt, A hybrid ensemble transform filter for nonlinear and spatially extended dynamical systems, SIAM/ASA J. Uncertainty Quantification, 4 (2016), 592-608.  doi: 10.1137/15M1040967.
    [9] D. Crisan and J. Xiong, Approximate McKean-Vlasov representation for a class of SPDEs, Stochastics, 82 (2010), 53-68.  doi: 10.1080/17442500902723575.
    [10] F. Daum and J. Huang, Particle filter for nonlinear filters, in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, (2011), 5920–5923.
    [11] J. de WiljesS. Reich and W. Stannat, Long-time stability and accuracy of the ensemble Kalman-Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), 1152-1181.  doi: 10.1137/17M1119056.
    [12] P. Degond and F.-J. Mustieles, A deterministic approximation of diffusion equations using particles, SIAM J. Sci. Comput., 11 (1990), 293-310.  doi: 10.1137/0911018.
    [13] G. DetommasoT. CuiA. SpantiniY. Marzouk and R. Scheichl, A Stein variational Newton method, Advances in Neural Information Processing Systems (NIPS 2018), 31 (2018), 9187-9197. 
    [14] A. A. Emerik and A. C. Reynolds, Ensemble smoother with multiple data assimilation, Computers & Geosciences, 55 (2013), 3-15.  doi: 10.1016/j.cageo.2012.03.011.
    [15] G. Evensen, Data Assimilation. The Ensemble Kalman Filter, Second edition, Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-642-03711-5.
    [16] A. Garbuno-Inigo, F. Hoffmann, W. Li and A. Stuart, Gradient Structure for the Ensemble Kalman Flow with Noise, Technical Report arXiv: 1903.08866.v2, Caltech, 2019.
    [17] O. Gonzalez, Time integration of discrete Hamiltonian systems, J. Nonlinear Sci., 6 (1996), 449-467.  doi: 10.1007/BF02440162.
    [18] J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Applied Mathematical Sciences, 160. Springer-Verlag, New York, 2005.
    [19] N. B. Kovachki and A. M. Stuart, Ensemble Kalman inversion: A derivative-free technique for machine learning tasks, Inverse Problems, 35 (2019), 095005, 35 pp, arXiv: 1808.03620.
    [20] K. Law, A. Stuart and K. Zygalakis, Data Assimilation: A Mathematical Introduction, Texts in Applied Mathematics, 62. Springer, Cham, 2015. doi: 10.1007/978-3-319-20325-6.
    [21] Q. Liu and D. Wang, Stein variational gradient descent: A general purpose Bayesian inference algorithm, in Advances in Neural Information Processing Systems (NIPS 2016), 29 (2016), 2378–2386.
    [22] E. Lorenz, Deterministic non-periodic flows, J. Atmos. Sci., 20 (1963), 130-141. 
    [23] R. I. McLachlanG. R. W. Quispel and N. Robidoux, Geometric integration using discrete gradients, Phil Trans. R. Soc. Lond. A Math. Phys. Eng. Sci., 357 (1999), 1021-1045.  doi: 10.1098/rsta.1999.0363.
    [24] J. Nocedal and S. J. Wright, Numerical Optimization, Second edition. Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006.
    [25] Y. Ollivier, Online natural gradient as a Kalman filter, Electronic Journal of Statistics, 12 (2018), 2930-2961.  doi: 10.1214/18-EJS1468.
    [26] G. Pavliotis, Stochastic Processes and Applications. Diffusion Processes, the Fokker-Planck and Langevin Equations, Texts in Applied Mathematics, 60. Springer, New York, 2014. doi: 10.1007/978-1-4939-1323-7.
    [27] S. Reich, Enhancing energy conserving methods, BIT, 36 (1996), 122-134.  doi: 10.1007/BF01740549.
    [28] S. Reich, A dynamical systems framework for intermittent data assimilation, BIT Numer Math, 51 (2011), 235-249.  doi: 10.1007/s10543-010-0302-4.
    [29] S. Reich, Data assimilation: The Schrödinger perspective, Acta Numerica, 28 (2019), 635-710.  doi: 10.1017/S0962492919000011.
    [30] S. Reich and  C. CotterProbabilistic Forecasting and Bayesian Data Assimilation, Cambridge University Press, Cambridge, 2015.  doi: 10.1017/CBO9781107706804.
    [31] C. P. Robert, The Bayesian Choice: From Decision-Theoretic Motivations to Computational Implementations, Second edition, Springer Texts in Statistics, Springer-Verlag, New York, 2001.
    [32] G. Russo, Deterministic diffusion of particles, Comm. Pure Appl. Math., 43 (1990), 697-733.  doi: 10.1002/cpa.3160430602.
    [33] P. Sakov and P. Oke, A deterministic formulation of the ensemble Kalman filter: An alternative to ensemble square root filters, Tellus, 60A (2008), 361-371. 
    [34] P. SakovD. Oliver and L. Bertino, An iterative EnKF for strongly nonlinear systems, Mon. Wea. Rev., 140 (2012), 1988-2004.  doi: 10.1175/MWR-D-11-00176.1.
    [35] A. M. Stuart, Numerical analysis and dynamical systems, Acta Numer., Cambridge Univ. Press, Cambridge, 3 (1994), 467–572. doi: 10.1017/S0962492900002488.
    [36] A. Tarantola, Inverse Problem Theory and Methods for Model Parameter Estimation, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2005. doi: 10.1137/1.9780898717921.
    [37] H. Yserentant, A new class of particle methods, Numer. Math., 76 (1997), 87-109.  doi: 10.1007/s002110050255.
  • 加载中

Figures(4)

Tables(1)

SHARE

Article Metrics

HTML views(671) PDF downloads(304) Cited by(0)

Access History

Other Articles By Authors

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return