American Institute of Mathematical Sciences

Advanced
  2020, 2(3): 333-349. doi: 10.3934/fods.2020016

Posterior contraction rates for non-parametric state and drift estimation

Sebastian Reich , and  Paul J. Rozdeba

Institute of Mathematics, University of Potsdam, Karl-Liebknecht-Str. 24/25, D-14476 Potsdam, Germany

* Corresponding author: Sebastian Reich

Received  March 2020 Revised  August 2020 Published   2020 Early access  October 2020

We consider a combined state and drift estimation problem for the linear stochastic heat equation. The infinite-dimensional Bayesian inference problem is formulated in terms of the Kalman–Bucy filter over an extended state space, and its long-time asymptotic properties are studied. Asymptotic posterior contraction rates in the unknown drift function are the main contribution of this paper. Such rates have been studied before for stationary non-parametric Bayesian inverse problems, and here we demonstrate the consistency of our time-dependent formulation with these previous results building upon scale separation and a slow manifold approximation.

Keywords: State and parameter estimation, Bayesian inference, posterior contraction rates, stochastic partial differential equations, Kalman–Bucy filter.
Mathematics Subject Classification: Primary: 62G20, 62G05, 62F15; Secondary: 60H15, 62M20.
Citation: Sebastian Reich, Paul J. Rozdeba. Posterior contraction rates for non-parametric state and drift estimation. Foundations of Data Science, 2020, 2 (3) : 333-349. doi: 10.3934/fods.2020016
References:
[1]

R. Altmeyer and M. Reiß, Nonparametric estimation for linear SPDEs from local measurements, Annals of Appl. Probability, preprint, arXiv: 1903.06984. Google Scholar

[2]

K. Bergemann and S. Reich, An ensemble Kalman–Bucy filter for continuous data assimilation, Meteorolog. Zeitschrift, 21 (2012), 213-219.  doi: 10.1127/0941-2948/2012/0307.  Google Scholar

[3]

I. Cialenco, Statistical inference for SPDEs: An overview, Stat. Inference Stoch. Process., 21 (2018), 309-329.  doi: 10.1007/s11203-018-9177-9.  Google Scholar

[4]

R. Curtain, A survey of infinite-dimensional filtering, SIAM Review, 17 (1975), 395-411.  doi: 10.1137/1017041.  Google Scholar

[5]

G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, 1st edition, Encyclopedia of Mathematics and its Applications, 45, Cambridge University Press, Cambridge, 1992. doi: 10.1017/CBO9780511666223.  Google Scholar

[6]

J. de Wiljes, S. Reich and W. Stannat, Long-time stability and accuracy of the ensemble Kalman–Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), 1152-1181.  doi: 10.1137/17M1119056.  Google Scholar

[7]

S. Ghosal and A. van der Vaart, Fundamentals of Nonparametric Bayesian Inference, Cambridge Series in Statistical and Probabilistic Mathematics, 44, Cambridge University Press, Cambridge, 2017. doi: 10.1017/9781139029834.  Google Scholar

[8]

E. Giné and R. Nickl, Mathematical Foundations of Infinite-Dimensional Statistical Models, Cambridge Series in Statistical and Probabilistic Mathematics, 40, Cambridge University Press, Cambridge, 2015. doi: 10.1017/CBO9781107337862.  Google Scholar

[9]

F. Götze, A. Naumov, V. Spokoiny and V. Ulyanov, Large ball probabilities, Gaussian comparison and anti-concentration, Bernoulli, 25 (2019), 2538-2563.  doi: 10.3150/18-BEJ1062.  Google Scholar

[10]

M. Hairer, An introduction to stochastic PDEs, unpublished lecture notes, arXiv: 0907.4178. Google Scholar

[11]

A. Jazwinski, Stochastic Processes and Filtering Theory, Mathematics in Science and Engineering, 64, Academic Press, New York, 1970. Google Scholar

[12]

B. T. Knapik, B. T. Szabó, A. W. van der Vaart and J. H. van Zanten, Bayes procedures for adaptive inference in inverse problems for the white noise model, Probab. Theory Relat. Fields, 164 (2016), 771-813.  doi: 10.1007/s00440-015-0619-7.  Google Scholar

[13]

B. T. Knapik, A. W. van der Vaart and J. H. van Zanten, Bayesian inverse problems with Gaussian priors, Annals of Statistics, 39 (2011), 2626-2657.  doi: 10.1214/11-AOS920.  Google Scholar

[14]

G. J. Lord, C. E. Powell and T. Shardlow, An Introduction to Computational Stochastic PDEs, 1st edition, Cambridge Texts in Applied Mathematics, 50, Cambridge University Press, Cambridge, 2014. doi: 10.1017/CBO9781139017329.  Google Scholar

[15]

N. Nüsken, S. Reich and P. J. Rozdeba, State and parameter estimation from observed signal increments, Entropy, 21 (2019), 505. doi: 10.3390/e21050505.  Google Scholar

[16]

E. Shchepakina, V. Sobolev and M. P. Mortell, Singular Perturbations: Introduction to System Order Reduction Methods with Applications, Lecture Notes in Mathematics, 2114, Springer International Publishing, 2014. doi: 10.1007/978-3-319-09570-7.  Google Scholar

[17]

D. Simon, Optimal State Estimation, Wiley, Hoboken, 2006. doi: 10.1002/0470045345.  Google Scholar

[18]

A. Taghvaei, J. de Wiljes, P. Mehta and S. Reich, Kalman filter and its modern extensions for the continuous-time nonlinear filtering problem, J. Dyn. Sys., Meas., Control., 140 (2017), 030904. doi: 10.1115/1.4037780.  Google Scholar

[19]

F. Verhulst, Singular perturbation methods for slow-fast dynamics, Nonlinear Dynamics, 50 (2007), 747-753.  doi: 10.1007/s11071-007-9236-z.  Google Scholar

[20]

D. Yan, Bayesian inference for Gaussian models: Inverse problems and evolution equations, PhD thesis, Universiteit Leiden, Leiden, 2020. doi: 1887/86070.  Google Scholar

[21]

T. Yang, P. Mehta and S. Meyn, Feedback particle filter, IEEE Trans. Automatic Control, 58 (2013), 2465-2480.  doi: 10.1109/TAC.2013.2258825.  Google Scholar

show all references

References:
[1]

R. Altmeyer and M. Reiß, Nonparametric estimation for linear SPDEs from local measurements, Annals of Appl. Probability, preprint, arXiv: 1903.06984. Google Scholar

[2]

K. Bergemann and S. Reich, An ensemble Kalman–Bucy filter for continuous data assimilation, Meteorolog. Zeitschrift, 21 (2012), 213-219.  doi: 10.1127/0941-2948/2012/0307.  Google Scholar

[3]

I. Cialenco, Statistical inference for SPDEs: An overview, Stat. Inference Stoch. Process., 21 (2018), 309-329.  doi: 10.1007/s11203-018-9177-9.  Google Scholar

[4]

R. Curtain, A survey of infinite-dimensional filtering, SIAM Review, 17 (1975), 395-411.  doi: 10.1137/1017041.  Google Scholar

[5]

G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, 1st edition, Encyclopedia of Mathematics and its Applications, 45, Cambridge University Press, Cambridge, 1992. doi: 10.1017/CBO9780511666223.  Google Scholar

[6]

J. de Wiljes, S. Reich and W. Stannat, Long-time stability and accuracy of the ensemble Kalman–Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), 1152-1181.  doi: 10.1137/17M1119056.  Google Scholar

[7]

S. Ghosal and A. van der Vaart, Fundamentals of Nonparametric Bayesian Inference, Cambridge Series in Statistical and Probabilistic Mathematics, 44, Cambridge University Press, Cambridge, 2017. doi: 10.1017/9781139029834.  Google Scholar

[8]

E. Giné and R. Nickl, Mathematical Foundations of Infinite-Dimensional Statistical Models, Cambridge Series in Statistical and Probabilistic Mathematics, 40, Cambridge University Press, Cambridge, 2015. doi: 10.1017/CBO9781107337862.  Google Scholar

[9]

F. Götze, A. Naumov, V. Spokoiny and V. Ulyanov, Large ball probabilities, Gaussian comparison and anti-concentration, Bernoulli, 25 (2019), 2538-2563.  doi: 10.3150/18-BEJ1062.  Google Scholar

[10]

M. Hairer, An introduction to stochastic PDEs, unpublished lecture notes, arXiv: 0907.4178. Google Scholar

[11]

A. Jazwinski, Stochastic Processes and Filtering Theory, Mathematics in Science and Engineering, 64, Academic Press, New York, 1970. Google Scholar

[12]

B. T. Knapik, B. T. Szabó, A. W. van der Vaart and J. H. van Zanten, Bayes procedures for adaptive inference in inverse problems for the white noise model, Probab. Theory Relat. Fields, 164 (2016), 771-813.  doi: 10.1007/s00440-015-0619-7.  Google Scholar

[13]

B. T. Knapik, A. W. van der Vaart and J. H. van Zanten, Bayesian inverse problems with Gaussian priors, Annals of Statistics, 39 (2011), 2626-2657.  doi: 10.1214/11-AOS920.  Google Scholar

[14]

G. J. Lord, C. E. Powell and T. Shardlow, An Introduction to Computational Stochastic PDEs, 1st edition, Cambridge Texts in Applied Mathematics, 50, Cambridge University Press, Cambridge, 2014. doi: 10.1017/CBO9781139017329.  Google Scholar

[15]

N. Nüsken, S. Reich and P. J. Rozdeba, State and parameter estimation from observed signal increments, Entropy, 21 (2019), 505. doi: 10.3390/e21050505.  Google Scholar

[16]

E. Shchepakina, V. Sobolev and M. P. Mortell, Singular Perturbations: Introduction to System Order Reduction Methods with Applications, Lecture Notes in Mathematics, 2114, Springer International Publishing, 2014. doi: 10.1007/978-3-319-09570-7.  Google Scholar

[17]

D. Simon, Optimal State Estimation, Wiley, Hoboken, 2006. doi: 10.1002/0470045345.  Google Scholar

[18]

A. Taghvaei, J. de Wiljes, P. Mehta and S. Reich, Kalman filter and its modern extensions for the continuous-time nonlinear filtering problem, J. Dyn. Sys., Meas., Control., 140 (2017), 030904. doi: 10.1115/1.4037780.  Google Scholar

[19]

F. Verhulst, Singular perturbation methods for slow-fast dynamics, Nonlinear Dynamics, 50 (2007), 747-753.  doi: 10.1007/s11071-007-9236-z.  Google Scholar

[20]

D. Yan, Bayesian inference for Gaussian models: Inverse problems and evolution equations, PhD thesis, Universiteit Leiden, Leiden, 2020. doi: 1887/86070.  Google Scholar

[21]

T. Yang, P. Mehta and S. Meyn, Feedback particle filter, IEEE Trans. Automatic Control, 58 (2013), 2465-2480.  doi: 10.1109/TAC.2013.2258825.  Google Scholar

Figure 1.  We display the time evolution of the bias $ |m_t^f(k)-\mathcal{F}^\ast(k)| $ and the variances $ \sigma_t^f(k) $ and $ p_t^f(k) $ for increasing values of $ k $. We compare the results from the combined state and parameter estimation problem (labelled 'dynamic') with those from the associated direct inference problem in the parameter alone (labelled 'stationary'). The delay in the onset of the asymptotic $ t^{-1} $ regime as $ k $ increases can be clearly seen as well as that $ p_t^f(k) < \sigma_t^f(k) $ for all $ t>0 $. Furthermore, as theoretically predicted, the dynamic and stationary problem formulations behave almost identically in terms of their drift estimates
Figure Options

Download full-size image

Download as PowerPoint slide

[1]

Kui Lin, Shuai Lu, Peter Mathé. Oracle-type posterior contraction rates in Bayesian inverse problems. Inverse Problems & Imaging, 2015, 9 (3) : 895-915. doi: 10.3934/ipi.2015.9.895
[2]

Russell Johnson, Carmen Núñez. The Kalman-Bucy filter revisited. Discrete & Continuous Dynamical Systems, 2014, 34 (10) : 4139-4153. doi: 10.3934/dcds.2014.34.4139
[3]

Jiangqi Wu, Linjie Wen, Jinglai Li. Resampled ensemble Kalman inversion for Bayesian parameter estimation with sequential data. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021045
[4]

Ferenc Hartung. Parameter estimation by quasilinearization in differential equations with state-dependent delays. Discrete & Continuous Dynamical Systems - B, 2013, 18 (6) : 1611-1631. doi: 10.3934/dcdsb.2013.18.1611
[5]

Laura Martín-Fernández, Gianni Gilioli, Ettore Lanzarone, Joaquín Míguez, Sara Pasquali, Fabrizio Ruggeri, Diego P. Ruiz. A Rao-Blackwellized particle filter for joint parameter estimation and biomass tracking in a stochastic predator-prey system. Mathematical Biosciences & Engineering, 2014, 11 (3) : 573-597. doi: 10.3934/mbe.2014.11.573
[6]

Junxiong Jia, Jigen Peng, Jinghuai Gao. Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption. Inverse Problems & Imaging, 2021, 15 (2) : 201-228. doi: 10.3934/ipi.2020061
[7]

Andrea Arnold, Daniela Calvetti, Erkki Somersalo. Vectorized and parallel particle filter SMC parameter estimation for stiff ODEs. Conference Publications, 2015, 2015 (special) : 75-84. doi: 10.3934/proc.2015.0075
[8]

Pingping Niu, Shuai Lu, Jin Cheng. On periodic parameter identification in stochastic differential equations. Inverse Problems & Imaging, 2019, 13 (3) : 513-543. doi: 10.3934/ipi.2019025
[9]

Junyoung Jang, Kihoon Jang, Hee-Dae Kwon, Jeehyun Lee. Feedback control of an HBV model based on ensemble kalman filter and differential evolution. Mathematical Biosciences & Engineering, 2018, 15 (3) : 667-691. doi: 10.3934/mbe.2018030
[10]

Lorenzo Zambotti. A brief and personal history of stochastic partial differential equations. Discrete & Continuous Dynamical Systems, 2021, 41 (1) : 471-487. doi: 10.3934/dcds.2020264
[11]

Arnulf Jentzen. Taylor expansions of solutions of stochastic partial differential equations. Discrete & Continuous Dynamical Systems - B, 2010, 14 (2) : 515-557. doi: 10.3934/dcdsb.2010.14.515
[12]

Robert Azencott, Yutheeka Gadhyan. Accurate parameter estimation for coupled stochastic dynamics. Conference Publications, 2009, 2009 (Special) : 44-53. doi: 10.3934/proc.2009.2009.44
[13]

Jin-Won Kim, Amirhossein Taghvaei, Yongxin Chen, Prashant G. Mehta. Feedback particle filter for collective inference. Foundations of Data Science, 2021, 3 (3) : 543-561. doi: 10.3934/fods.2021018
[14]

Deng Lu, Maria De Iorio, Ajay Jasra, Gary L. Rosner. Bayesian inference for latent chain graphs. Foundations of Data Science, 2020, 2 (1) : 35-54. doi: 10.3934/fods.2020003
[15]

Sahani Pathiraja, Sebastian Reich. Discrete gradients for computational Bayesian inference. Journal of Computational Dynamics, 2019, 6 (2) : 385-400. doi: 10.3934/jcd.2019019
[16]

Dominique Chapelle, Philippe Moireau, Patrick Le Tallec. Robust filtering for joint state-parameter estimation in distributed mechanical systems. Discrete & Continuous Dynamical Systems, 2009, 23 (1&2) : 65-84. doi: 10.3934/dcds.2009.23.65
[17]

Alexander Bibov, Heikki Haario, Antti Solonen. Stabilized BFGS approximate Kalman filter. Inverse Problems & Imaging, 2015, 9 (4) : 1003-1024. doi: 10.3934/ipi.2015.9.1003
[18]

Min Yang, Guanggan Chen. Finite dimensional reducing and smooth approximating for a class of stochastic partial differential equations. Discrete & Continuous Dynamical Systems - B, 2020, 25 (4) : 1565-1581. doi: 10.3934/dcdsb.2019240
[19]

Qi Lü, Xu Zhang. A concise introduction to control theory for stochastic partial differential equations. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021020
[20]

Tomás Caraballo, José Real, T. Taniguchi. The exponential stability of neutral stochastic delay partial differential equations. Discrete & Continuous Dynamical Systems, 2007, 18 (2&3) : 295-313. doi: 10.3934/dcds.2007.18.295

 Impact Factor: 

Tools

Metrics

  • PDF downloads (121)
  • HTML views (340)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences | Privacy Policy