American Institute of Mathematical Sciences

Advanced
August  2015, 9(3): 895-915. doi: 10.3934/ipi.2015.9.895

Oracle-type posterior contraction rates in Bayesian inverse problems

Kui Lin 1, , Shuai Lu 1, and  Peter Mathé 2,

1. 

School of Mathematical Sciences, Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University, Shanghai, 200433, China, China
2. 

Weierstraß Institute for Applied Analysis and Stochastics, Mohrenstraße 39, 10117 Berlin, Germany

Received  April 2014 Revised  June 2015 Published  July 2015

We discuss Bayesian inverse problems in Hilbert spaces. The focus is on a fast concentration of the posterior probability around the unknown true solution as expressed in the concept of posterior contraction rates. This concentration is dominated by a parameter which controls the variance of the prior distribution. Previous results determine posterior contraction rates based on known solution smoothness. Here we show that an oracle-type parameter choice is possible. This is done by relating the posterior contraction rate to the root mean squared estimation error. In addition we show that the tail probability, which usually is bounded by using the Chebyshev inequality, has exponential decay, at least for a priori parameter choices. These results implement the exponential concentration of Gaussian measures in Hilbert spaces.
Keywords: effective dimension., posterior contraction, Bayesian inverse problem.
Mathematics Subject Classification: Primary: 62F15; Secondary: 65J20, 65J22, 62C1.
Citation: Kui Lin, Shuai Lu, Peter Mathé. Oracle-type posterior contraction rates in Bayesian inverse problems. Inverse Problems and Imaging, 2015, 9 (3) : 895-915. doi: 10.3934/ipi.2015.9.895
References:
[1]

S. Agapiou, Aspects of Bayesian Inverse Problems, Ph.D. thesis, University of Warwick, 2013.

[2]

S. Agapiou, S. Larsson and A. M. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems, Stochastic Process. Appl., 123 (2013), 3828-3860. doi: 10.1016/j.spa.2013.05.001.

[3]

S. Agapiou and P. Mathé, Preconditioning the prior to avoid saturation in Bayesian inverse problems, submitted, arXiv:1409.6496, 2014.

[4]

F. Bauer and M. A. Lukas, Comparing parameter choice methods for regularization of ill-posed problems, Math. Comput. Simulation, 81 (2011), 1795-1841. doi: 10.1016/j.matcom.2011.01.016.

[5]

N. Bissantz, T. Hohage and A. Munk, Consistency and rates of convergence of nonlinear Tikhonov regularization with random noise, Inverse Problems, 20 (2004), 1773-1789. doi: 10.1088/0266-5611/20/6/005.

[6]

G. Blanchard and P. Mathé, Discrepancy principle for statistical inverse problems with application to conjugate gradient regularization, Inverse Problems, 28 (2012), 115011, 23pp. doi: 10.1088/0266-5611/28/11/115011.

[7]

A. Caponnetto and E. De Vito, Optimal rates for the regularized least-squares algorithm, Found. Comput. Math., 7 (2007), 331-368. doi: 10.1007/s10208-006-0196-8.

[8]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Mathematics and its Applications, 375, Kluwer Academic Publishers Group, Dordrecht, 1996. doi: 10.1007/978-94-009-1740-8.

[9]

B. G. Fitzpatrick, Bayesian analysis in inverse problems, Inverse Problems, 7 (1991), 675-702. doi: 10.1088/0266-5611/7/5/003.

[10]

J.-P. Florens and A. Simoni, Regularizing priors for linear inverse problems, Scand. J. Stat., 39 (2012), 214-235. doi: 10.1111/j.1467-9469.2011.00784.x.

[11]

P. C. Hansen, Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems, Numer. Algorithms, 6 (1994), 1-35. doi: 10.1007/BF02149761.

[12]

J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Applied Mathematical Sciences, 160, Springer-Verlag, New York, 2005.

[13]

B. T. Knapik, A. W. van der Vaart and H. van Zanten, Bayesian inverse problems with Gaussian priors, Ann. Statist., 39 (2011), 2626-2657. doi: 10.1214/11-AOS920.

[14]

S. Lasanen, Non-Gaussian statistical inverse problems. Part I: Posterior distributions, Inverse Probl. Imaging, 6 (2012), 215-266. doi: 10.3934/ipi.2012.6.215.

[15]

S. Lasanen, Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns, Inverse Probl. Imaging, 6 (2012), 267-287. doi: 10.3934/ipi.2012.6.267.

[16]

M. Lassas, E. Saksman and S. Siltanen, Discretization-invariant Bayesian inversion and Besov space priors, Inverse Problems and Imaging, 3 (2009), 87-122. doi: 10.3934/ipi.2009.3.87.

[17]

M. Lassas and S. Siltanen, Can one use total variation prior for edge-preserving Bayesian inversion?, Inverse Problems, 20 (2004), 1537-1563. doi: 10.1088/0266-5611/20/5/013.

[18]

M. Ledoux and M. Talagrand, Probability in Banach Spaces. Isoperimetry and Processes, Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)], 23, Springer-Verlag, Berlin, 1991. doi: 10.1007/978-3-642-20212-4.

[19]

M. S. Lehtinen, L. Päivärinta and E. Somersalo, Linear inverse problems for generalised random variables, Inverse Problems, 5 (1989), 599-612. doi: 10.1088/0266-5611/5/4/011.

[20]

S. Lu and S. V. Pereverzev, Regularization Theory for Ill-Posed Problems. SElected Topics, Inverse and Ill-posed Problems Series, 58, De Gruyter, Berlin, 2013. doi: 10.1515/9783110286496.

[21]

S. Lu and P. Mathé, Discrepancy based model selection in statistical inverse problems, J. Complexity, 30 (2014), 290-308. doi: 10.1016/j.jco.2014.02.002.

[22]

A. Mandelbaum, Linear estimators and measurable linear transformations on a Hilbert space, Z. Wahrsch. Verw. Gebiete, 65 (1984), 385-397. doi: 10.1007/BF00533743.

[23]

C. Marteau and P. Mathé, General regularization schemes for signal detection in inverse problems, Math. Meth. Statist., 23 (2014), 176-200. doi: 10.3103/S1066530714030028.

[24]

P. Mathé and B. Hofmann, How general are general source conditions?, Inverse Problems, 24 (2008), 015009, 5pp. doi: 10.1088/0266-5611/24/1/015009.

[25]

P. Mathé and S. V. Pereverzev, Geometry of linear ill-posed problems in variable Hilbert scales, Inverse Problems, 19 (2003), 789-803. doi: 10.1088/0266-5611/19/3/319.

[26]

P. Mathé and S. V. Pereverzev, Regularization of some linear ill-posed problems with discretized random noisy data, Math. Comp., 75 (2006), 1913-1929. doi: 10.1090/S0025-5718-06-01873-4.

[27]

P. Mathé and U. Tautenhahn, Enhancing linear regularization to treat large noise, J. Inverse Ill-Posed Probl., 19 (2011), 859-879. doi: 10.1515/JIIP.2011.052.

[28]

A. M. Stuart, Inverse problems: A Bayesian perspective, Acta Numer., 19 (2010), 451-559. doi: 10.1017/S0962492910000061.

show all references

References:
[1]

S. Agapiou, Aspects of Bayesian Inverse Problems, Ph.D. thesis, University of Warwick, 2013.

[2]

S. Agapiou, S. Larsson and A. M. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems, Stochastic Process. Appl., 123 (2013), 3828-3860. doi: 10.1016/j.spa.2013.05.001.

[3]

S. Agapiou and P. Mathé, Preconditioning the prior to avoid saturation in Bayesian inverse problems, submitted, arXiv:1409.6496, 2014.

[4]

F. Bauer and M. A. Lukas, Comparing parameter choice methods for regularization of ill-posed problems, Math. Comput. Simulation, 81 (2011), 1795-1841. doi: 10.1016/j.matcom.2011.01.016.

[5]

N. Bissantz, T. Hohage and A. Munk, Consistency and rates of convergence of nonlinear Tikhonov regularization with random noise, Inverse Problems, 20 (2004), 1773-1789. doi: 10.1088/0266-5611/20/6/005.

[6]

G. Blanchard and P. Mathé, Discrepancy principle for statistical inverse problems with application to conjugate gradient regularization, Inverse Problems, 28 (2012), 115011, 23pp. doi: 10.1088/0266-5611/28/11/115011.

[7]

A. Caponnetto and E. De Vito, Optimal rates for the regularized least-squares algorithm, Found. Comput. Math., 7 (2007), 331-368. doi: 10.1007/s10208-006-0196-8.

[8]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Mathematics and its Applications, 375, Kluwer Academic Publishers Group, Dordrecht, 1996. doi: 10.1007/978-94-009-1740-8.

[9]

B. G. Fitzpatrick, Bayesian analysis in inverse problems, Inverse Problems, 7 (1991), 675-702. doi: 10.1088/0266-5611/7/5/003.

[10]

J.-P. Florens and A. Simoni, Regularizing priors for linear inverse problems, Scand. J. Stat., 39 (2012), 214-235. doi: 10.1111/j.1467-9469.2011.00784.x.

[11]

P. C. Hansen, Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems, Numer. Algorithms, 6 (1994), 1-35. doi: 10.1007/BF02149761.

[12]

J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Applied Mathematical Sciences, 160, Springer-Verlag, New York, 2005.

[13]

B. T. Knapik, A. W. van der Vaart and H. van Zanten, Bayesian inverse problems with Gaussian priors, Ann. Statist., 39 (2011), 2626-2657. doi: 10.1214/11-AOS920.

[14]

S. Lasanen, Non-Gaussian statistical inverse problems. Part I: Posterior distributions, Inverse Probl. Imaging, 6 (2012), 215-266. doi: 10.3934/ipi.2012.6.215.

[15]

S. Lasanen, Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns, Inverse Probl. Imaging, 6 (2012), 267-287. doi: 10.3934/ipi.2012.6.267.

[16]

M. Lassas, E. Saksman and S. Siltanen, Discretization-invariant Bayesian inversion and Besov space priors, Inverse Problems and Imaging, 3 (2009), 87-122. doi: 10.3934/ipi.2009.3.87.

[17]

M. Lassas and S. Siltanen, Can one use total variation prior for edge-preserving Bayesian inversion?, Inverse Problems, 20 (2004), 1537-1563. doi: 10.1088/0266-5611/20/5/013.

[18]

M. Ledoux and M. Talagrand, Probability in Banach Spaces. Isoperimetry and Processes, Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)], 23, Springer-Verlag, Berlin, 1991. doi: 10.1007/978-3-642-20212-4.

[19]

M. S. Lehtinen, L. Päivärinta and E. Somersalo, Linear inverse problems for generalised random variables, Inverse Problems, 5 (1989), 599-612. doi: 10.1088/0266-5611/5/4/011.

[20]

S. Lu and S. V. Pereverzev, Regularization Theory for Ill-Posed Problems. SElected Topics, Inverse and Ill-posed Problems Series, 58, De Gruyter, Berlin, 2013. doi: 10.1515/9783110286496.

[21]

S. Lu and P. Mathé, Discrepancy based model selection in statistical inverse problems, J. Complexity, 30 (2014), 290-308. doi: 10.1016/j.jco.2014.02.002.

[22]

A. Mandelbaum, Linear estimators and measurable linear transformations on a Hilbert space, Z. Wahrsch. Verw. Gebiete, 65 (1984), 385-397. doi: 10.1007/BF00533743.

[23]

C. Marteau and P. Mathé, General regularization schemes for signal detection in inverse problems, Math. Meth. Statist., 23 (2014), 176-200. doi: 10.3103/S1066530714030028.

[24]

P. Mathé and B. Hofmann, How general are general source conditions?, Inverse Problems, 24 (2008), 015009, 5pp. doi: 10.1088/0266-5611/24/1/015009.

[25]

P. Mathé and S. V. Pereverzev, Geometry of linear ill-posed problems in variable Hilbert scales, Inverse Problems, 19 (2003), 789-803. doi: 10.1088/0266-5611/19/3/319.

[26]

P. Mathé and S. V. Pereverzev, Regularization of some linear ill-posed problems with discretized random noisy data, Math. Comp., 75 (2006), 1913-1929. doi: 10.1090/S0025-5718-06-01873-4.

[27]

P. Mathé and U. Tautenhahn, Enhancing linear regularization to treat large noise, J. Inverse Ill-Posed Probl., 19 (2011), 859-879. doi: 10.1515/JIIP.2011.052.

[28]

A. M. Stuart, Inverse problems: A Bayesian perspective, Acta Numer., 19 (2010), 451-559. doi: 10.1017/S0962492910000061.

[1]

Junxiong Jia, Jigen Peng, Jinghuai Gao. Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption. Inverse Problems and Imaging, 2021, 15 (2) : 201-228. doi: 10.3934/ipi.2020061
[2]

Sebastian Reich, Paul J. Rozdeba. Posterior contraction rates for non-parametric state and drift estimation. Foundations of Data Science, 2020, 2 (3) : 333-349. doi: 10.3934/fods.2020016
[3]

Jiangfeng Huang, Zhiliang Deng, Liwei Xu. A Bayesian level set method for an inverse medium scattering problem in acoustics. Inverse Problems and Imaging, 2021, 15 (5) : 1077-1097. doi: 10.3934/ipi.2021029
[4]

Yunwen Yin, Weishi Yin, Pinchao Meng, Hongyu Liu. The interior inverse scattering problem for a two-layered cavity using the Bayesian method. Inverse Problems and Imaging, , () : -. doi: 10.3934/ipi.2021069
[5]

Masoumeh Dashti, Stephen Harris, Andrew Stuart. Besov priors for Bayesian inverse problems. Inverse Problems and Imaging, 2012, 6 (2) : 183-200. doi: 10.3934/ipi.2012.6.183
[6]

Guillaume Bal, Ian Langmore, Youssef Marzouk. Bayesian inverse problems with Monte Carlo forward models. Inverse Problems and Imaging, 2013, 7 (1) : 81-105. doi: 10.3934/ipi.2013.7.81
[7]

Sari Lasanen. Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns. Inverse Problems and Imaging, 2012, 6 (2) : 267-287. doi: 10.3934/ipi.2012.6.267
[8]

Sari Lasanen. Non-Gaussian statistical inverse problems. Part I: Posterior distributions. Inverse Problems and Imaging, 2012, 6 (2) : 215-266. doi: 10.3934/ipi.2012.6.215
[9]

Tan Bui-Thanh, Omar Ghattas. A scalable algorithm for MAP estimators in Bayesian inverse problems with Besov priors. Inverse Problems and Imaging, 2015, 9 (1) : 27-53. doi: 10.3934/ipi.2015.9.27
[10]

Victor Isakov, Joseph Myers. On the inverse doping profile problem. Inverse Problems and Imaging, 2012, 6 (3) : 465-486. doi: 10.3934/ipi.2012.6.465
[11]

Michael L. Frankel, Victor Roytburd. Fractal dimension of attractors for a Stefan problem. Conference Publications, 2003, 2003 (Special) : 281-287. doi: 10.3934/proc.2003.2003.281
[12]

Lincoln Chayes, Inwon C. Kim. The supercooled Stefan problem in one dimension. Communications on Pure and Applied Analysis, 2012, 11 (2) : 845-859. doi: 10.3934/cpaa.2012.11.845
[13]

Tan Bui-Thanh, Quoc P. Nguyen. FEM-based discretization-invariant MCMC methods for PDE-constrained Bayesian inverse problems. Inverse Problems and Imaging, 2016, 10 (4) : 943-975. doi: 10.3934/ipi.2016028
[14]

T. J. Sullivan. Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors. Inverse Problems and Imaging, 2017, 11 (5) : 857-874. doi: 10.3934/ipi.2017040
[15]

Didi Lv, Qingping Zhou, Jae Kyu Choi, Jinglai Li, Xiaoqun Zhang. Nonlocal TV-Gaussian prior for Bayesian inverse problems with applications to limited CT reconstruction. Inverse Problems and Imaging, 2020, 14 (1) : 117-132. doi: 10.3934/ipi.2019066
[16]

Meixin Xiong, Liuhong Chen, Ju Ming, Jaemin Shin. Accelerating the Bayesian inference of inverse problems by using data-driven compressive sensing method based on proper orthogonal decomposition. Electronic Research Archive, 2021, 29 (5) : 3383-3403. doi: 10.3934/era.2021044
[17]

Kazuhiro Kawamura. Mean dimension of shifts of finite type and of generalized inverse limits. Discrete and Continuous Dynamical Systems, 2020, 40 (8) : 4767-4775. doi: 10.3934/dcds.2020200
[18]

Alexandr Mikhaylov, Victor Mikhaylov. Dynamic inverse problem for Jacobi matrices. Inverse Problems and Imaging, 2019, 13 (3) : 431-447. doi: 10.3934/ipi.2019021
[19]

Armin Lechleiter, Tobias Rienmüller. Factorization method for the inverse Stokes problem. Inverse Problems and Imaging, 2013, 7 (4) : 1271-1293. doi: 10.3934/ipi.2013.7.1271
[20]

Nguyen Huy Tuan, Mokhtar Kirane, Long Dinh Le, Van Thinh Nguyen. On an inverse problem for fractional evolution equation. Evolution Equations and Control Theory, 2017, 6 (1) : 111-134. doi: 10.3934/eect.2017007

2020 Impact Factor: 1.639

Tools

Metrics

  • PDF downloads (151)
  • HTML views (0)
  • Cited by (9)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy