American Institute of Mathematical Sciences

Advanced
  2020, 2(3): 333-349. doi: 10.3934/fods.2020016

Posterior contraction rates for non-parametric state and drift estimation

Sebastian Reich , and  Paul J. Rozdeba

Institute of Mathematics, University of Potsdam, Karl-Liebknecht-Str. 24/25, D-14476 Potsdam, Germany

* Corresponding author: Sebastian Reich

Received  March 2020 Revised  August 2020 Published   2020 Early access  October 2020

We consider a combined state and drift estimation problem for the linear stochastic heat equation. The infinite-dimensional Bayesian inference problem is formulated in terms of the Kalman–Bucy filter over an extended state space, and its long-time asymptotic properties are studied. Asymptotic posterior contraction rates in the unknown drift function are the main contribution of this paper. Such rates have been studied before for stationary non-parametric Bayesian inverse problems, and here we demonstrate the consistency of our time-dependent formulation with these previous results building upon scale separation and a slow manifold approximation.

Keywords: State and parameter estimation, Bayesian inference, posterior contraction rates, stochastic partial differential equations, Kalman–Bucy filter.
Mathematics Subject Classification: Primary: 62G20, 62G05, 62F15; Secondary: 60H15, 62M20.
Citation: Sebastian Reich, Paul J. Rozdeba. Posterior contraction rates for non-parametric state and drift estimation. Foundations of Data Science, 2020, 2 (3) : 333-349. doi: 10.3934/fods.2020016
References:
[1]

R. Altmeyer and M. Reiß, Nonparametric estimation for linear SPDEs from local measurements, Annals of Appl. Probability, preprint, arXiv: 1903.06984.

[2]

K. Bergemann and S. Reich, An ensemble Kalman–Bucy filter for continuous data assimilation, Meteorolog. Zeitschrift, 21 (2012), 213-219.  doi: 10.1127/0941-2948/2012/0307.

[3]

I. Cialenco, Statistical inference for SPDEs: An overview, Stat. Inference Stoch. Process., 21 (2018), 309-329.  doi: 10.1007/s11203-018-9177-9.

[4]

R. Curtain, A survey of infinite-dimensional filtering, SIAM Review, 17 (1975), 395-411.  doi: 10.1137/1017041.

[5]

G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, 1st edition, Encyclopedia of Mathematics and its Applications, 45, Cambridge University Press, Cambridge, 1992. doi: 10.1017/CBO9780511666223.

[6]

J. de Wiljes, S. Reich and W. Stannat, Long-time stability and accuracy of the ensemble Kalman–Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), 1152-1181.  doi: 10.1137/17M1119056.

[7]

S. Ghosal and A. van der Vaart, Fundamentals of Nonparametric Bayesian Inference, Cambridge Series in Statistical and Probabilistic Mathematics, 44, Cambridge University Press, Cambridge, 2017. doi: 10.1017/9781139029834.

[8]

E. Giné and R. Nickl, Mathematical Foundations of Infinite-Dimensional Statistical Models, Cambridge Series in Statistical and Probabilistic Mathematics, 40, Cambridge University Press, Cambridge, 2015. doi: 10.1017/CBO9781107337862.

[9]

F. Götze, A. Naumov, V. Spokoiny and V. Ulyanov, Large ball probabilities, Gaussian comparison and anti-concentration, Bernoulli, 25 (2019), 2538-2563.  doi: 10.3150/18-BEJ1062.

[10]

M. Hairer, An introduction to stochastic PDEs, unpublished lecture notes, arXiv: 0907.4178.

[11]

A. Jazwinski, Stochastic Processes and Filtering Theory, Mathematics in Science and Engineering, 64, Academic Press, New York, 1970.

[12]

B. T. Knapik, B. T. Szabó, A. W. van der Vaart and J. H. van Zanten, Bayes procedures for adaptive inference in inverse problems for the white noise model, Probab. Theory Relat. Fields, 164 (2016), 771-813.  doi: 10.1007/s00440-015-0619-7.

[13]

B. T. Knapik, A. W. van der Vaart and J. H. van Zanten, Bayesian inverse problems with Gaussian priors, Annals of Statistics, 39 (2011), 2626-2657.  doi: 10.1214/11-AOS920.

[14]

G. J. Lord, C. E. Powell and T. Shardlow, An Introduction to Computational Stochastic PDEs, 1st edition, Cambridge Texts in Applied Mathematics, 50, Cambridge University Press, Cambridge, 2014. doi: 10.1017/CBO9781139017329.

[15]

N. Nüsken, S. Reich and P. J. Rozdeba, State and parameter estimation from observed signal increments, Entropy, 21 (2019), 505. doi: 10.3390/e21050505.

[16]

E. Shchepakina, V. Sobolev and M. P. Mortell, Singular Perturbations: Introduction to System Order Reduction Methods with Applications, Lecture Notes in Mathematics, 2114, Springer International Publishing, 2014. doi: 10.1007/978-3-319-09570-7.

[17]

D. Simon, Optimal State Estimation, Wiley, Hoboken, 2006. doi: 10.1002/0470045345.

[18]

A. Taghvaei, J. de Wiljes, P. Mehta and S. Reich, Kalman filter and its modern extensions for the continuous-time nonlinear filtering problem, J. Dyn. Sys., Meas., Control., 140 (2017), 030904. doi: 10.1115/1.4037780.

[19]

F. Verhulst, Singular perturbation methods for slow-fast dynamics, Nonlinear Dynamics, 50 (2007), 747-753.  doi: 10.1007/s11071-007-9236-z.

[20]

D. Yan, Bayesian inference for Gaussian models: Inverse problems and evolution equations, PhD thesis, Universiteit Leiden, Leiden, 2020. doi: 1887/86070.

[21]

T. Yang, P. Mehta and S. Meyn, Feedback particle filter, IEEE Trans. Automatic Control, 58 (2013), 2465-2480.  doi: 10.1109/TAC.2013.2258825.

show all references

References:
[1]

R. Altmeyer and M. Reiß, Nonparametric estimation for linear SPDEs from local measurements, Annals of Appl. Probability, preprint, arXiv: 1903.06984.

[2]

K. Bergemann and S. Reich, An ensemble Kalman–Bucy filter for continuous data assimilation, Meteorolog. Zeitschrift, 21 (2012), 213-219.  doi: 10.1127/0941-2948/2012/0307.

[3]

I. Cialenco, Statistical inference for SPDEs: An overview, Stat. Inference Stoch. Process., 21 (2018), 309-329.  doi: 10.1007/s11203-018-9177-9.

[4]

R. Curtain, A survey of infinite-dimensional filtering, SIAM Review, 17 (1975), 395-411.  doi: 10.1137/1017041.

[5]

G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, 1st edition, Encyclopedia of Mathematics and its Applications, 45, Cambridge University Press, Cambridge, 1992. doi: 10.1017/CBO9780511666223.

[6]

J. de Wiljes, S. Reich and W. Stannat, Long-time stability and accuracy of the ensemble Kalman–Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), 1152-1181.  doi: 10.1137/17M1119056.

[7]

S. Ghosal and A. van der Vaart, Fundamentals of Nonparametric Bayesian Inference, Cambridge Series in Statistical and Probabilistic Mathematics, 44, Cambridge University Press, Cambridge, 2017. doi: 10.1017/9781139029834.

[8]

E. Giné and R. Nickl, Mathematical Foundations of Infinite-Dimensional Statistical Models, Cambridge Series in Statistical and Probabilistic Mathematics, 40, Cambridge University Press, Cambridge, 2015. doi: 10.1017/CBO9781107337862.

[9]

F. Götze, A. Naumov, V. Spokoiny and V. Ulyanov, Large ball probabilities, Gaussian comparison and anti-concentration, Bernoulli, 25 (2019), 2538-2563.  doi: 10.3150/18-BEJ1062.

[10]

M. Hairer, An introduction to stochastic PDEs, unpublished lecture notes, arXiv: 0907.4178.

[11]

A. Jazwinski, Stochastic Processes and Filtering Theory, Mathematics in Science and Engineering, 64, Academic Press, New York, 1970.

[12]

B. T. Knapik, B. T. Szabó, A. W. van der Vaart and J. H. van Zanten, Bayes procedures for adaptive inference in inverse problems for the white noise model, Probab. Theory Relat. Fields, 164 (2016), 771-813.  doi: 10.1007/s00440-015-0619-7.

[13]

B. T. Knapik, A. W. van der Vaart and J. H. van Zanten, Bayesian inverse problems with Gaussian priors, Annals of Statistics, 39 (2011), 2626-2657.  doi: 10.1214/11-AOS920.

[14]

G. J. Lord, C. E. Powell and T. Shardlow, An Introduction to Computational Stochastic PDEs, 1st edition, Cambridge Texts in Applied Mathematics, 50, Cambridge University Press, Cambridge, 2014. doi: 10.1017/CBO9781139017329.

[15]

N. Nüsken, S. Reich and P. J. Rozdeba, State and parameter estimation from observed signal increments, Entropy, 21 (2019), 505. doi: 10.3390/e21050505.

[16]

E. Shchepakina, V. Sobolev and M. P. Mortell, Singular Perturbations: Introduction to System Order Reduction Methods with Applications, Lecture Notes in Mathematics, 2114, Springer International Publishing, 2014. doi: 10.1007/978-3-319-09570-7.

[17]

D. Simon, Optimal State Estimation, Wiley, Hoboken, 2006. doi: 10.1002/0470045345.

[18]

A. Taghvaei, J. de Wiljes, P. Mehta and S. Reich, Kalman filter and its modern extensions for the continuous-time nonlinear filtering problem, J. Dyn. Sys., Meas., Control., 140 (2017), 030904. doi: 10.1115/1.4037780.

[19]

F. Verhulst, Singular perturbation methods for slow-fast dynamics, Nonlinear Dynamics, 50 (2007), 747-753.  doi: 10.1007/s11071-007-9236-z.

[20]

D. Yan, Bayesian inference for Gaussian models: Inverse problems and evolution equations, PhD thesis, Universiteit Leiden, Leiden, 2020. doi: 1887/86070.

[21]

T. Yang, P. Mehta and S. Meyn, Feedback particle filter, IEEE Trans. Automatic Control, 58 (2013), 2465-2480.  doi: 10.1109/TAC.2013.2258825.

Figure 1.  We display the time evolution of the bias $ |m_t^f(k)-\mathcal{F}^\ast(k)| $ and the variances $ \sigma_t^f(k) $ and $ p_t^f(k) $ for increasing values of $ k $. We compare the results from the combined state and parameter estimation problem (labelled 'dynamic') with those from the associated direct inference problem in the parameter alone (labelled 'stationary'). The delay in the onset of the asymptotic $ t^{-1} $ regime as $ k $ increases can be clearly seen as well as that $ p_t^f(k) < \sigma_t^f(k) $ for all $ t>0 $. Furthermore, as theoretically predicted, the dynamic and stationary problem formulations behave almost identically in terms of their drift estimates
Figure Options

Download full-size image

Download as PowerPoint slide

[1]

Kui Lin, Shuai Lu, Peter Mathé. Oracle-type posterior contraction rates in Bayesian inverse problems. Inverse Problems and Imaging, 2015, 9 (3) : 895-915. doi: 10.3934/ipi.2015.9.895
[2]

Russell Johnson, Carmen Núñez. The Kalman-Bucy filter revisited. Discrete and Continuous Dynamical Systems, 2014, 34 (10) : 4139-4153. doi: 10.3934/dcds.2014.34.4139
[3]

Jiangqi Wu, Linjie Wen, Jinglai Li. Resampled ensemble Kalman inversion for Bayesian parameter estimation with sequential data. Discrete and Continuous Dynamical Systems - S, 2022, 15 (4) : 837-850. doi: 10.3934/dcdss.2021045
[4]

Ferenc Hartung. Parameter estimation by quasilinearization in differential equations with state-dependent delays. Discrete and Continuous Dynamical Systems - B, 2013, 18 (6) : 1611-1631. doi: 10.3934/dcdsb.2013.18.1611
[5]

Laura Martín-Fernández, Gianni Gilioli, Ettore Lanzarone, Joaquín Míguez, Sara Pasquali, Fabrizio Ruggeri, Diego P. Ruiz. A Rao-Blackwellized particle filter for joint parameter estimation and biomass tracking in a stochastic predator-prey system. Mathematical Biosciences & Engineering, 2014, 11 (3) : 573-597. doi: 10.3934/mbe.2014.11.573
[6]

Junxiong Jia, Jigen Peng, Jinghuai Gao. Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption. Inverse Problems and Imaging, 2021, 15 (2) : 201-228. doi: 10.3934/ipi.2020061
[7]

Andrea Arnold, Daniela Calvetti, Erkki Somersalo. Vectorized and parallel particle filter SMC parameter estimation for stiff ODEs. Conference Publications, 2015, 2015 (special) : 75-84. doi: 10.3934/proc.2015.0075
[8]

Pingping Niu, Shuai Lu, Jin Cheng. On periodic parameter identification in stochastic differential equations. Inverse Problems and Imaging, 2019, 13 (3) : 513-543. doi: 10.3934/ipi.2019025
[9]

Junyoung Jang, Kihoon Jang, Hee-Dae Kwon, Jeehyun Lee. Feedback control of an HBV model based on ensemble kalman filter and differential evolution. Mathematical Biosciences & Engineering, 2018, 15 (3) : 667-691. doi: 10.3934/mbe.2018030
[10]

Lorenzo Zambotti. A brief and personal history of stochastic partial differential equations. Discrete and Continuous Dynamical Systems, 2021, 41 (1) : 471-487. doi: 10.3934/dcds.2020264
[11]

Arnulf Jentzen. Taylor expansions of solutions of stochastic partial differential equations. Discrete and Continuous Dynamical Systems - B, 2010, 14 (2) : 515-557. doi: 10.3934/dcdsb.2010.14.515
[12]

Robert Azencott, Yutheeka Gadhyan. Accurate parameter estimation for coupled stochastic dynamics. Conference Publications, 2009, 2009 (Special) : 44-53. doi: 10.3934/proc.2009.2009.44
[13]

Yanyan Hu, Fubao Xi, Min Zhu. Least squares estimation for distribution-dependent stochastic differential delay equations. Communications on Pure and Applied Analysis, 2022, 21 (4) : 1505-1536. doi: 10.3934/cpaa.2022027
[14]

Jin-Won Kim, Amirhossein Taghvaei, Yongxin Chen, Prashant G. Mehta. Feedback particle filter for collective inference. Foundations of Data Science, 2021, 3 (3) : 543-561. doi: 10.3934/fods.2021018
[15]

Deng Lu, Maria De Iorio, Ajay Jasra, Gary L. Rosner. Bayesian inference for latent chain graphs. Foundations of Data Science, 2020, 2 (1) : 35-54. doi: 10.3934/fods.2020003
[16]

Sahani Pathiraja, Sebastian Reich. Discrete gradients for computational Bayesian inference. Journal of Computational Dynamics, 2019, 6 (2) : 385-400. doi: 10.3934/jcd.2019019
[17]

Dominique Chapelle, Philippe Moireau, Patrick Le Tallec. Robust filtering for joint state-parameter estimation in distributed mechanical systems. Discrete and Continuous Dynamical Systems, 2009, 23 (1&2) : 65-84. doi: 10.3934/dcds.2009.23.65
[18]

Alexander Bibov, Heikki Haario, Antti Solonen. Stabilized BFGS approximate Kalman filter. Inverse Problems and Imaging, 2015, 9 (4) : 1003-1024. doi: 10.3934/ipi.2015.9.1003
[19]

Min Yang, Guanggan Chen. Finite dimensional reducing and smooth approximating for a class of stochastic partial differential equations. Discrete and Continuous Dynamical Systems - B, 2020, 25 (4) : 1565-1581. doi: 10.3934/dcdsb.2019240
[20]

Qi Lü, Xu Zhang. A concise introduction to control theory for stochastic partial differential equations. Mathematical Control and Related Fields, 2021  doi: 10.3934/mcrf.2021020

 Impact Factor: 

Tools

Metrics

  • PDF downloads (163)
  • HTML views (459)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy