
-
Previous Article
Geometry of the Kahan discretizations of planar quadratic Hamiltonian systems. Ⅱ. Systems with a linear Poisson tensor
- JCD Home
- This Issue
-
Next Article
A structure-preserving Fourier pseudo-spectral linearly implicit scheme for the space-fractional nonlinear Schrödinger equation
Discrete gradients for computational Bayesian inference
University of Potsdam, Institute of Mathematics, Karl-Liebknecht-Str. 24/25, Potsdam, D-14476, Germany |
In this paper, we exploit the gradient flow structure of continuous-time formulations of Bayesian inference in terms of their numerical time-stepping. We focus on two particular examples, namely, the continuous-time ensemble Kalman–Bucy filter and a particle discretisation of the Fokker–Planck equation associated to Brownian dynamics. Both formulations can lead to stiff differential equations which require special numerical methods for their efficient numerical implementation. We compare discrete gradient methods to alternative semi-implicit and other iterative implementations of the underlying Bayesian inference problems.
References:
[1] |
J. Amezcua, E. Kalnay, K. Ide and S. Reich,
Ensemble transform Kalman-Bucy filters, Q. J. R. Meteor. Soc., 140 (2014), 995-1004.
doi: 10.1002/qj.2186. |
[2] |
U. M. Ascher, Numerical Methods for Evolutionary Differential Equations, Computational Science & Engineering, 5. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2008.
doi: 10.1137/1.9780898718911. |
[3] |
K. Bergemann and S. Reich,
A localization technique for ensemble Kalman filters, Q. J. R. Meteorological Soc., 136 (2010), 701-707.
doi: 10.1002/qj.591. |
[4] |
K. Bergemann and S. Reich,
A mollified ensemble Kalman filter, Q. J. R. Meteorological Soc., 136 (2010), 1636-1643.
doi: 10.1002/qj.672. |
[5] |
K. Bergemann and S. Reich,
An ensemble Kalman-Bucy filter for continuous data assimilation, Meteorolog. Zeitschrift, 21 (2012), 213-219.
doi: 10.1127/0941-2948/2012/0307. |
[6] |
D. Blömker, C. Schillings and P. Wacker,
A strongly convergent numerical scheme for ensemble Kalman inversion, SIAM J. Numer. Anal., 56 (2018), 2537-2562.
doi: 10.1137/17M1132367. |
[7] |
Y. Chen and D. S. Oliver,
Levenberg-Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification, Computational Geoscience, 17 (2013), 689-703.
doi: 10.1007/s10596-013-9351-5. |
[8] |
N. Chustagulprom, S. Reich and M. Reinhardt,
A hybrid ensemble transform filter for nonlinear and spatially extended dynamical systems, SIAM/ASA J. Uncertainty Quantification, 4 (2016), 592-608.
doi: 10.1137/15M1040967. |
[9] |
D. Crisan and J. Xiong,
Approximate McKean-Vlasov representation for a class of SPDEs, Stochastics, 82 (2010), 53-68.
doi: 10.1080/17442500902723575. |
[10] |
F. Daum and J. Huang, Particle filter for nonlinear filters, in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, (2011), 5920–5923. Google Scholar |
[11] |
J. de Wiljes, S. Reich and W. Stannat,
Long-time stability and accuracy of the ensemble Kalman-Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), 1152-1181.
doi: 10.1137/17M1119056. |
[12] |
P. Degond and F.-J. Mustieles,
A deterministic approximation of diffusion equations using particles, SIAM J. Sci. Comput., 11 (1990), 293-310.
doi: 10.1137/0911018. |
[13] |
G. Detommaso, T. Cui, A. Spantini, Y. Marzouk and R. Scheichl, A Stein variational Newton method, Advances in Neural Information Processing Systems (NIPS 2018), 31 (2018), 9187-9197. Google Scholar |
[14] |
A. A. Emerik and A. C. Reynolds,
Ensemble smoother with multiple data assimilation, Computers & Geosciences, 55 (2013), 3-15.
doi: 10.1016/j.cageo.2012.03.011. |
[15] |
G. Evensen, Data Assimilation. The Ensemble Kalman Filter, Second edition, Springer-Verlag, Berlin, 2009.
doi: 10.1007/978-3-642-03711-5. |
[16] |
A. Garbuno-Inigo, F. Hoffmann, W. Li and A. Stuart, Gradient Structure for the Ensemble Kalman Flow with Noise, Technical Report arXiv: 1903.08866.v2, Caltech, 2019. Google Scholar |
[17] |
O. Gonzalez,
Time integration of discrete Hamiltonian systems, J. Nonlinear Sci., 6 (1996), 449-467.
doi: 10.1007/BF02440162. |
[18] |
J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Applied Mathematical Sciences, 160. Springer-Verlag, New York, 2005. |
[19] |
N. B. Kovachki and A. M. Stuart, Ensemble Kalman inversion: A derivative-free technique for machine learning tasks, Inverse Problems, 35 (2019), 095005, 35 pp, arXiv: 1808.03620. |
[20] |
K. Law, A. Stuart and K. Zygalakis, Data Assimilation: A Mathematical Introduction, Texts in Applied Mathematics, 62. Springer, Cham, 2015.
doi: 10.1007/978-3-319-20325-6. |
[21] |
Q. Liu and D. Wang, Stein variational gradient descent: A general purpose Bayesian inference algorithm, in Advances in Neural Information Processing Systems (NIPS 2016), 29 (2016), 2378–2386. Google Scholar |
[22] |
E. Lorenz, Deterministic non-periodic flows, J. Atmos. Sci., 20 (1963), 130-141. Google Scholar |
[23] |
R. I. McLachlan, G. R. W. Quispel and N. Robidoux,
Geometric integration using discrete gradients, Phil Trans. R. Soc. Lond. A Math. Phys. Eng. Sci., 357 (1999), 1021-1045.
doi: 10.1098/rsta.1999.0363. |
[24] |
J. Nocedal and S. J. Wright, Numerical Optimization, Second edition. Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006. |
[25] |
Y. Ollivier,
Online natural gradient as a Kalman filter, Electronic Journal of Statistics, 12 (2018), 2930-2961.
doi: 10.1214/18-EJS1468. |
[26] |
G. Pavliotis, Stochastic Processes and Applications. Diffusion Processes, the Fokker-Planck and Langevin Equations, Texts in Applied Mathematics, 60. Springer, New York, 2014.
doi: 10.1007/978-1-4939-1323-7. |
[27] |
S. Reich,
Enhancing energy conserving methods, BIT, 36 (1996), 122-134.
doi: 10.1007/BF01740549. |
[28] |
S. Reich,
A dynamical systems framework for intermittent data assimilation, BIT Numer Math, 51 (2011), 235-249.
doi: 10.1007/s10543-010-0302-4. |
[29] |
S. Reich,
Data assimilation: The Schrödinger perspective, Acta Numerica, 28 (2019), 635-710.
doi: 10.1017/S0962492919000011. |
[30] |
S. Reich and C. Cotter, Probabilistic Forecasting and Bayesian Data Assimilation, Cambridge University Press, Cambridge, 2015.
doi: 10.1017/CBO9781107706804.![]() ![]() |
[31] |
C. P. Robert, The Bayesian Choice: From Decision-Theoretic Motivations to Computational Implementations, Second edition, Springer Texts in Statistics, Springer-Verlag, New York, 2001. |
[32] |
G. Russo,
Deterministic diffusion of particles, Comm. Pure Appl. Math., 43 (1990), 697-733.
doi: 10.1002/cpa.3160430602. |
[33] |
P. Sakov and P. Oke, A deterministic formulation of the ensemble Kalman filter: An alternative to ensemble square root filters, Tellus, 60A (2008), 361-371. Google Scholar |
[34] |
P. Sakov, D. Oliver and L. Bertino,
An iterative EnKF for strongly nonlinear systems, Mon. Wea. Rev., 140 (2012), 1988-2004.
doi: 10.1175/MWR-D-11-00176.1. |
[35] |
A. M. Stuart, Numerical analysis and dynamical systems, Acta Numer., Cambridge Univ. Press, Cambridge, 3 (1994), 467–572.
doi: 10.1017/S0962492900002488. |
[36] |
A. Tarantola, Inverse Problem Theory and Methods for Model Parameter Estimation, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2005.
doi: 10.1137/1.9780898717921. |
[37] |
H. Yserentant,
A new class of particle methods, Numer. Math., 76 (1997), 87-109.
doi: 10.1007/s002110050255. |
show all references
References:
[1] |
J. Amezcua, E. Kalnay, K. Ide and S. Reich,
Ensemble transform Kalman-Bucy filters, Q. J. R. Meteor. Soc., 140 (2014), 995-1004.
doi: 10.1002/qj.2186. |
[2] |
U. M. Ascher, Numerical Methods for Evolutionary Differential Equations, Computational Science & Engineering, 5. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2008.
doi: 10.1137/1.9780898718911. |
[3] |
K. Bergemann and S. Reich,
A localization technique for ensemble Kalman filters, Q. J. R. Meteorological Soc., 136 (2010), 701-707.
doi: 10.1002/qj.591. |
[4] |
K. Bergemann and S. Reich,
A mollified ensemble Kalman filter, Q. J. R. Meteorological Soc., 136 (2010), 1636-1643.
doi: 10.1002/qj.672. |
[5] |
K. Bergemann and S. Reich,
An ensemble Kalman-Bucy filter for continuous data assimilation, Meteorolog. Zeitschrift, 21 (2012), 213-219.
doi: 10.1127/0941-2948/2012/0307. |
[6] |
D. Blömker, C. Schillings and P. Wacker,
A strongly convergent numerical scheme for ensemble Kalman inversion, SIAM J. Numer. Anal., 56 (2018), 2537-2562.
doi: 10.1137/17M1132367. |
[7] |
Y. Chen and D. S. Oliver,
Levenberg-Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification, Computational Geoscience, 17 (2013), 689-703.
doi: 10.1007/s10596-013-9351-5. |
[8] |
N. Chustagulprom, S. Reich and M. Reinhardt,
A hybrid ensemble transform filter for nonlinear and spatially extended dynamical systems, SIAM/ASA J. Uncertainty Quantification, 4 (2016), 592-608.
doi: 10.1137/15M1040967. |
[9] |
D. Crisan and J. Xiong,
Approximate McKean-Vlasov representation for a class of SPDEs, Stochastics, 82 (2010), 53-68.
doi: 10.1080/17442500902723575. |
[10] |
F. Daum and J. Huang, Particle filter for nonlinear filters, in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, (2011), 5920–5923. Google Scholar |
[11] |
J. de Wiljes, S. Reich and W. Stannat,
Long-time stability and accuracy of the ensemble Kalman-Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), 1152-1181.
doi: 10.1137/17M1119056. |
[12] |
P. Degond and F.-J. Mustieles,
A deterministic approximation of diffusion equations using particles, SIAM J. Sci. Comput., 11 (1990), 293-310.
doi: 10.1137/0911018. |
[13] |
G. Detommaso, T. Cui, A. Spantini, Y. Marzouk and R. Scheichl, A Stein variational Newton method, Advances in Neural Information Processing Systems (NIPS 2018), 31 (2018), 9187-9197. Google Scholar |
[14] |
A. A. Emerik and A. C. Reynolds,
Ensemble smoother with multiple data assimilation, Computers & Geosciences, 55 (2013), 3-15.
doi: 10.1016/j.cageo.2012.03.011. |
[15] |
G. Evensen, Data Assimilation. The Ensemble Kalman Filter, Second edition, Springer-Verlag, Berlin, 2009.
doi: 10.1007/978-3-642-03711-5. |
[16] |
A. Garbuno-Inigo, F. Hoffmann, W. Li and A. Stuart, Gradient Structure for the Ensemble Kalman Flow with Noise, Technical Report arXiv: 1903.08866.v2, Caltech, 2019. Google Scholar |
[17] |
O. Gonzalez,
Time integration of discrete Hamiltonian systems, J. Nonlinear Sci., 6 (1996), 449-467.
doi: 10.1007/BF02440162. |
[18] |
J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Applied Mathematical Sciences, 160. Springer-Verlag, New York, 2005. |
[19] |
N. B. Kovachki and A. M. Stuart, Ensemble Kalman inversion: A derivative-free technique for machine learning tasks, Inverse Problems, 35 (2019), 095005, 35 pp, arXiv: 1808.03620. |
[20] |
K. Law, A. Stuart and K. Zygalakis, Data Assimilation: A Mathematical Introduction, Texts in Applied Mathematics, 62. Springer, Cham, 2015.
doi: 10.1007/978-3-319-20325-6. |
[21] |
Q. Liu and D. Wang, Stein variational gradient descent: A general purpose Bayesian inference algorithm, in Advances in Neural Information Processing Systems (NIPS 2016), 29 (2016), 2378–2386. Google Scholar |
[22] |
E. Lorenz, Deterministic non-periodic flows, J. Atmos. Sci., 20 (1963), 130-141. Google Scholar |
[23] |
R. I. McLachlan, G. R. W. Quispel and N. Robidoux,
Geometric integration using discrete gradients, Phil Trans. R. Soc. Lond. A Math. Phys. Eng. Sci., 357 (1999), 1021-1045.
doi: 10.1098/rsta.1999.0363. |
[24] |
J. Nocedal and S. J. Wright, Numerical Optimization, Second edition. Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006. |
[25] |
Y. Ollivier,
Online natural gradient as a Kalman filter, Electronic Journal of Statistics, 12 (2018), 2930-2961.
doi: 10.1214/18-EJS1468. |
[26] |
G. Pavliotis, Stochastic Processes and Applications. Diffusion Processes, the Fokker-Planck and Langevin Equations, Texts in Applied Mathematics, 60. Springer, New York, 2014.
doi: 10.1007/978-1-4939-1323-7. |
[27] |
S. Reich,
Enhancing energy conserving methods, BIT, 36 (1996), 122-134.
doi: 10.1007/BF01740549. |
[28] |
S. Reich,
A dynamical systems framework for intermittent data assimilation, BIT Numer Math, 51 (2011), 235-249.
doi: 10.1007/s10543-010-0302-4. |
[29] |
S. Reich,
Data assimilation: The Schrödinger perspective, Acta Numerica, 28 (2019), 635-710.
doi: 10.1017/S0962492919000011. |
[30] |
S. Reich and C. Cotter, Probabilistic Forecasting and Bayesian Data Assimilation, Cambridge University Press, Cambridge, 2015.
doi: 10.1017/CBO9781107706804.![]() ![]() |
[31] |
C. P. Robert, The Bayesian Choice: From Decision-Theoretic Motivations to Computational Implementations, Second edition, Springer Texts in Statistics, Springer-Verlag, New York, 2001. |
[32] |
G. Russo,
Deterministic diffusion of particles, Comm. Pure Appl. Math., 43 (1990), 697-733.
doi: 10.1002/cpa.3160430602. |
[33] |
P. Sakov and P. Oke, A deterministic formulation of the ensemble Kalman filter: An alternative to ensemble square root filters, Tellus, 60A (2008), 361-371. Google Scholar |
[34] |
P. Sakov, D. Oliver and L. Bertino,
An iterative EnKF for strongly nonlinear systems, Mon. Wea. Rev., 140 (2012), 1988-2004.
doi: 10.1175/MWR-D-11-00176.1. |
[35] |
A. M. Stuart, Numerical analysis and dynamical systems, Acta Numer., Cambridge Univ. Press, Cambridge, 3 (1994), 467–572.
doi: 10.1017/S0962492900002488. |
[36] |
A. Tarantola, Inverse Problem Theory and Methods for Model Parameter Estimation, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2005.
doi: 10.1137/1.9780898717921. |
[37] |
H. Yserentant,
A new class of particle methods, Numer. Math., 76 (1997), 87-109.
doi: 10.1007/s002110050255. |




[1] |
Guillaume Bal, Ian Langmore, Youssef Marzouk. Bayesian inverse problems with Monte Carlo forward models. Inverse Problems & Imaging, 2013, 7 (1) : 81-105. doi: 10.3934/ipi.2013.7.81 |
[2] |
Deng Lu, Maria De Iorio, Ajay Jasra, Gary L. Rosner. Bayesian inference for latent chain graphs. Foundations of Data Science, 2020, 2 (1) : 35-54. doi: 10.3934/fods.2020003 |
[3] |
Giacomo Dimarco. The moment guided Monte Carlo method for the Boltzmann equation. Kinetic & Related Models, 2013, 6 (2) : 291-315. doi: 10.3934/krm.2013.6.291 |
[4] |
Ajay Jasra, Kody J. H. Law, Yaxian Xu. Markov chain simulation for multilevel Monte Carlo. Foundations of Data Science, 2021 doi: 10.3934/fods.2021004 |
[5] |
Marc Bocquet, Julien Brajard, Alberto Carrassi, Laurent Bertino. Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization. Foundations of Data Science, 2020, 2 (1) : 55-80. doi: 10.3934/fods.2020004 |
[6] |
Evangelos Evangelou. Approximate Bayesian inference for geostatistical generalised linear models. Foundations of Data Science, 2019, 1 (1) : 39-60. doi: 10.3934/fods.2019002 |
[7] |
Michael B. Giles, Kristian Debrabant, Andreas Rössler. Analysis of multilevel Monte Carlo path simulation using the Milstein discretisation. Discrete & Continuous Dynamical Systems - B, 2019, 24 (8) : 3881-3903. doi: 10.3934/dcdsb.2018335 |
[8] |
Jiakou Wang, Margaret J. Slattery, Meghan Henty Hoskins, Shile Liang, Cheng Dong, Qiang Du. Monte carlo simulation of heterotypic cell aggregation in nonlinear shear flow. Mathematical Biosciences & Engineering, 2006, 3 (4) : 683-696. doi: 10.3934/mbe.2006.3.683 |
[9] |
Domokos Szász. Algebro-geometric methods for hard ball systems. Discrete & Continuous Dynamical Systems - A, 2008, 22 (1&2) : 427-443. doi: 10.3934/dcds.2008.22.427 |
[10] |
Yanyan Shi, Yajuan Sun, Yulei Wang, Jian Liu. Study of adaptive symplectic methods for simulating charged particle dynamics. Journal of Computational Dynamics, 2019, 6 (2) : 429-448. doi: 10.3934/jcd.2019022 |
[11] |
Joseph Nebus. The Dirichlet quotient of point vortex interactions on the surface of the sphere examined by Monte Carlo experiments. Discrete & Continuous Dynamical Systems - B, 2005, 5 (1) : 125-136. doi: 10.3934/dcdsb.2005.5.125 |
[12] |
Chjan C. Lim, Joseph Nebus, Syed M. Assad. Monte-Carlo and polyhedron-based simulations I: extremal states of the logarithmic N-body problem on a sphere. Discrete & Continuous Dynamical Systems - B, 2003, 3 (3) : 313-342. doi: 10.3934/dcdsb.2003.3.313 |
[13] |
Olli-Pekka Tossavainen, Daniel B. Work. Markov Chain Monte Carlo based inverse modeling of traffic flows using GPS data. Networks & Heterogeneous Media, 2013, 8 (3) : 803-824. doi: 10.3934/nhm.2013.8.803 |
[14] |
Mazyar Zahedi-Seresht, Gholam-Reza Jahanshahloo, Josef Jablonsky, Sedighe Asghariniya. A new Monte Carlo based procedure for complete ranking efficient units in DEA models. Numerical Algebra, Control & Optimization, 2017, 7 (4) : 403-416. doi: 10.3934/naco.2017025 |
[15] |
Nicolas Forcadel, Cyril Imbert, Régis Monneau. Homogenization of some particle systems with two-body interactions and of the dislocation dynamics. Discrete & Continuous Dynamical Systems - A, 2009, 23 (3) : 785-826. doi: 10.3934/dcds.2009.23.785 |
[16] |
Doron Levy, Tiago Requeijo. Modeling group dynamics of phototaxis: From particle systems to PDEs. Discrete & Continuous Dynamical Systems - B, 2008, 9 (1) : 103-128. doi: 10.3934/dcdsb.2008.9.103 |
[17] |
Oliver J. Maclaren, Helen M. Byrne, Alexander G. Fletcher, Philip K. Maini. Models, measurement and inference in epithelial tissue dynamics. Mathematical Biosciences & Engineering, 2015, 12 (6) : 1321-1340. doi: 10.3934/mbe.2015.12.1321 |
[18] |
Luca Dieci, Cinzia Elia. Smooth to discontinuous systems: A geometric and numerical method for slow-fast dynamics. Discrete & Continuous Dynamical Systems - B, 2018, 23 (7) : 2935-2950. doi: 10.3934/dcdsb.2018112 |
[19] |
Vadim Kaushansky, Christoph Reisinger. Simulation of a simple particle system interacting through hitting times. Discrete & Continuous Dynamical Systems - B, 2019, 24 (10) : 5481-5502. doi: 10.3934/dcdsb.2019067 |
[20] |
Yuhong Dai, Ya-xiang Yuan. Analysis of monotone gradient methods. Journal of Industrial & Management Optimization, 2005, 1 (2) : 181-192. doi: 10.3934/jimo.2005.1.181 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]