American Institute of Mathematical Sciences

Advanced
August  2015, 9(3): 895-915. doi: 10.3934/ipi.2015.9.895

Oracle-type posterior contraction rates in Bayesian inverse problems

Kui Lin 1, , Shuai Lu 1, and  Peter Mathé 2,

1. 

School of Mathematical Sciences, Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University, Shanghai, 200433, China, China
2. 

Weierstraß Institute for Applied Analysis and Stochastics, Mohrenstraße 39, 10117 Berlin, Germany

Received  April 2014 Revised  June 2015 Published  July 2015

We discuss Bayesian inverse problems in Hilbert spaces. The focus is on a fast concentration of the posterior probability around the unknown true solution as expressed in the concept of posterior contraction rates. This concentration is dominated by a parameter which controls the variance of the prior distribution. Previous results determine posterior contraction rates based on known solution smoothness. Here we show that an oracle-type parameter choice is possible. This is done by relating the posterior contraction rate to the root mean squared estimation error. In addition we show that the tail probability, which usually is bounded by using the Chebyshev inequality, has exponential decay, at least for a priori parameter choices. These results implement the exponential concentration of Gaussian measures in Hilbert spaces.
Keywords: effective dimension., posterior contraction, Bayesian inverse problem.
Mathematics Subject Classification: Primary: 62F15; Secondary: 65J20, 65J22, 62C1.
Citation: Kui Lin, Shuai Lu, Peter Mathé. Oracle-type posterior contraction rates in Bayesian inverse problems. Inverse Problems & Imaging, 2015, 9 (3) : 895-915. doi: 10.3934/ipi.2015.9.895
References:
[1]

S. Agapiou, Aspects of Bayesian Inverse Problems, Ph.D. thesis, University of Warwick, 2013. Google Scholar

[2]

S. Agapiou, S. Larsson and A. M. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems, Stochastic Process. Appl., 123 (2013), 3828-3860. doi: 10.1016/j.spa.2013.05.001.  Google Scholar

[3]

S. Agapiou and P. Mathé, Preconditioning the prior to avoid saturation in Bayesian inverse problems, submitted, arXiv:1409.6496, 2014. Google Scholar

[4]

F. Bauer and M. A. Lukas, Comparing parameter choice methods for regularization of ill-posed problems, Math. Comput. Simulation, 81 (2011), 1795-1841. doi: 10.1016/j.matcom.2011.01.016.  Google Scholar

[5]

N. Bissantz, T. Hohage and A. Munk, Consistency and rates of convergence of nonlinear Tikhonov regularization with random noise, Inverse Problems, 20 (2004), 1773-1789. doi: 10.1088/0266-5611/20/6/005.  Google Scholar

[6]

G. Blanchard and P. Mathé, Discrepancy principle for statistical inverse problems with application to conjugate gradient regularization, Inverse Problems, 28 (2012), 115011, 23pp. doi: 10.1088/0266-5611/28/11/115011.  Google Scholar

[7]

A. Caponnetto and E. De Vito, Optimal rates for the regularized least-squares algorithm, Found. Comput. Math., 7 (2007), 331-368. doi: 10.1007/s10208-006-0196-8.  Google Scholar

[8]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Mathematics and its Applications, 375, Kluwer Academic Publishers Group, Dordrecht, 1996. doi: 10.1007/978-94-009-1740-8.  Google Scholar

[9]

B. G. Fitzpatrick, Bayesian analysis in inverse problems, Inverse Problems, 7 (1991), 675-702. doi: 10.1088/0266-5611/7/5/003.  Google Scholar

[10]

J.-P. Florens and A. Simoni, Regularizing priors for linear inverse problems, Scand. J. Stat., 39 (2012), 214-235. doi: 10.1111/j.1467-9469.2011.00784.x.  Google Scholar

[11]

P. C. Hansen, Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems, Numer. Algorithms, 6 (1994), 1-35. doi: 10.1007/BF02149761.  Google Scholar

[12]

J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Applied Mathematical Sciences, 160, Springer-Verlag, New York, 2005.  Google Scholar

[13]

B. T. Knapik, A. W. van der Vaart and H. van Zanten, Bayesian inverse problems with Gaussian priors, Ann. Statist., 39 (2011), 2626-2657. doi: 10.1214/11-AOS920.  Google Scholar

[14]

S. Lasanen, Non-Gaussian statistical inverse problems. Part I: Posterior distributions, Inverse Probl. Imaging, 6 (2012), 215-266. doi: 10.3934/ipi.2012.6.215.  Google Scholar

[15]

S. Lasanen, Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns, Inverse Probl. Imaging, 6 (2012), 267-287. doi: 10.3934/ipi.2012.6.267.  Google Scholar

[16]

M. Lassas, E. Saksman and S. Siltanen, Discretization-invariant Bayesian inversion and Besov space priors, Inverse Problems and Imaging, 3 (2009), 87-122. doi: 10.3934/ipi.2009.3.87.  Google Scholar

[17]

M. Lassas and S. Siltanen, Can one use total variation prior for edge-preserving Bayesian inversion?, Inverse Problems, 20 (2004), 1537-1563. doi: 10.1088/0266-5611/20/5/013.  Google Scholar

[18]

M. Ledoux and M. Talagrand, Probability in Banach Spaces. Isoperimetry and Processes, Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)], 23, Springer-Verlag, Berlin, 1991. doi: 10.1007/978-3-642-20212-4.  Google Scholar

[19]

M. S. Lehtinen, L. Päivärinta and E. Somersalo, Linear inverse problems for generalised random variables, Inverse Problems, 5 (1989), 599-612. doi: 10.1088/0266-5611/5/4/011.  Google Scholar

[20]

S. Lu and S. V. Pereverzev, Regularization Theory for Ill-Posed Problems. SElected Topics, Inverse and Ill-posed Problems Series, 58, De Gruyter, Berlin, 2013. doi: 10.1515/9783110286496.  Google Scholar

[21]

S. Lu and P. Mathé, Discrepancy based model selection in statistical inverse problems, J. Complexity, 30 (2014), 290-308. doi: 10.1016/j.jco.2014.02.002.  Google Scholar

[22]

A. Mandelbaum, Linear estimators and measurable linear transformations on a Hilbert space, Z. Wahrsch. Verw. Gebiete, 65 (1984), 385-397. doi: 10.1007/BF00533743.  Google Scholar

[23]

C. Marteau and P. Mathé, General regularization schemes for signal detection in inverse problems, Math. Meth. Statist., 23 (2014), 176-200. doi: 10.3103/S1066530714030028.  Google Scholar

[24]

P. Mathé and B. Hofmann, How general are general source conditions?, Inverse Problems, 24 (2008), 015009, 5pp. doi: 10.1088/0266-5611/24/1/015009.  Google Scholar

[25]

P. Mathé and S. V. Pereverzev, Geometry of linear ill-posed problems in variable Hilbert scales, Inverse Problems, 19 (2003), 789-803. doi: 10.1088/0266-5611/19/3/319.  Google Scholar

[26]

P. Mathé and S. V. Pereverzev, Regularization of some linear ill-posed problems with discretized random noisy data, Math. Comp., 75 (2006), 1913-1929. doi: 10.1090/S0025-5718-06-01873-4.  Google Scholar

[27]

P. Mathé and U. Tautenhahn, Enhancing linear regularization to treat large noise, J. Inverse Ill-Posed Probl., 19 (2011), 859-879. doi: 10.1515/JIIP.2011.052.  Google Scholar

[28]

A. M. Stuart, Inverse problems: A Bayesian perspective, Acta Numer., 19 (2010), 451-559. doi: 10.1017/S0962492910000061.  Google Scholar

show all references

References:
[1]

S. Agapiou, Aspects of Bayesian Inverse Problems, Ph.D. thesis, University of Warwick, 2013. Google Scholar

[2]

S. Agapiou, S. Larsson and A. M. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems, Stochastic Process. Appl., 123 (2013), 3828-3860. doi: 10.1016/j.spa.2013.05.001.  Google Scholar

[3]

S. Agapiou and P. Mathé, Preconditioning the prior to avoid saturation in Bayesian inverse problems, submitted, arXiv:1409.6496, 2014. Google Scholar

[4]

F. Bauer and M. A. Lukas, Comparing parameter choice methods for regularization of ill-posed problems, Math. Comput. Simulation, 81 (2011), 1795-1841. doi: 10.1016/j.matcom.2011.01.016.  Google Scholar

[5]

N. Bissantz, T. Hohage and A. Munk, Consistency and rates of convergence of nonlinear Tikhonov regularization with random noise, Inverse Problems, 20 (2004), 1773-1789. doi: 10.1088/0266-5611/20/6/005.  Google Scholar

[6]

G. Blanchard and P. Mathé, Discrepancy principle for statistical inverse problems with application to conjugate gradient regularization, Inverse Problems, 28 (2012), 115011, 23pp. doi: 10.1088/0266-5611/28/11/115011.  Google Scholar

[7]

A. Caponnetto and E. De Vito, Optimal rates for the regularized least-squares algorithm, Found. Comput. Math., 7 (2007), 331-368. doi: 10.1007/s10208-006-0196-8.  Google Scholar

[8]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Mathematics and its Applications, 375, Kluwer Academic Publishers Group, Dordrecht, 1996. doi: 10.1007/978-94-009-1740-8.  Google Scholar

[9]

B. G. Fitzpatrick, Bayesian analysis in inverse problems, Inverse Problems, 7 (1991), 675-702. doi: 10.1088/0266-5611/7/5/003.  Google Scholar

[10]

J.-P. Florens and A. Simoni, Regularizing priors for linear inverse problems, Scand. J. Stat., 39 (2012), 214-235. doi: 10.1111/j.1467-9469.2011.00784.x.  Google Scholar

[11]

P. C. Hansen, Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems, Numer. Algorithms, 6 (1994), 1-35. doi: 10.1007/BF02149761.  Google Scholar

[12]

J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Applied Mathematical Sciences, 160, Springer-Verlag, New York, 2005.  Google Scholar

[13]

B. T. Knapik, A. W. van der Vaart and H. van Zanten, Bayesian inverse problems with Gaussian priors, Ann. Statist., 39 (2011), 2626-2657. doi: 10.1214/11-AOS920.  Google Scholar

[14]

S. Lasanen, Non-Gaussian statistical inverse problems. Part I: Posterior distributions, Inverse Probl. Imaging, 6 (2012), 215-266. doi: 10.3934/ipi.2012.6.215.  Google Scholar

[15]

S. Lasanen, Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns, Inverse Probl. Imaging, 6 (2012), 267-287. doi: 10.3934/ipi.2012.6.267.  Google Scholar

[16]

M. Lassas, E. Saksman and S. Siltanen, Discretization-invariant Bayesian inversion and Besov space priors, Inverse Problems and Imaging, 3 (2009), 87-122. doi: 10.3934/ipi.2009.3.87.  Google Scholar

[17]

M. Lassas and S. Siltanen, Can one use total variation prior for edge-preserving Bayesian inversion?, Inverse Problems, 20 (2004), 1537-1563. doi: 10.1088/0266-5611/20/5/013.  Google Scholar

[18]

M. Ledoux and M. Talagrand, Probability in Banach Spaces. Isoperimetry and Processes, Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)], 23, Springer-Verlag, Berlin, 1991. doi: 10.1007/978-3-642-20212-4.  Google Scholar

[19]

M. S. Lehtinen, L. Päivärinta and E. Somersalo, Linear inverse problems for generalised random variables, Inverse Problems, 5 (1989), 599-612. doi: 10.1088/0266-5611/5/4/011.  Google Scholar

[20]

S. Lu and S. V. Pereverzev, Regularization Theory for Ill-Posed Problems. SElected Topics, Inverse and Ill-posed Problems Series, 58, De Gruyter, Berlin, 2013. doi: 10.1515/9783110286496.  Google Scholar

[21]

S. Lu and P. Mathé, Discrepancy based model selection in statistical inverse problems, J. Complexity, 30 (2014), 290-308. doi: 10.1016/j.jco.2014.02.002.  Google Scholar

[22]

A. Mandelbaum, Linear estimators and measurable linear transformations on a Hilbert space, Z. Wahrsch. Verw. Gebiete, 65 (1984), 385-397. doi: 10.1007/BF00533743.  Google Scholar

[23]

C. Marteau and P. Mathé, General regularization schemes for signal detection in inverse problems, Math. Meth. Statist., 23 (2014), 176-200. doi: 10.3103/S1066530714030028.  Google Scholar

[24]

P. Mathé and B. Hofmann, How general are general source conditions?, Inverse Problems, 24 (2008), 015009, 5pp. doi: 10.1088/0266-5611/24/1/015009.  Google Scholar

[25]

P. Mathé and S. V. Pereverzev, Geometry of linear ill-posed problems in variable Hilbert scales, Inverse Problems, 19 (2003), 789-803. doi: 10.1088/0266-5611/19/3/319.  Google Scholar

[26]

P. Mathé and S. V. Pereverzev, Regularization of some linear ill-posed problems with discretized random noisy data, Math. Comp., 75 (2006), 1913-1929. doi: 10.1090/S0025-5718-06-01873-4.  Google Scholar

[27]

P. Mathé and U. Tautenhahn, Enhancing linear regularization to treat large noise, J. Inverse Ill-Posed Probl., 19 (2011), 859-879. doi: 10.1515/JIIP.2011.052.  Google Scholar

[28]

A. M. Stuart, Inverse problems: A Bayesian perspective, Acta Numer., 19 (2010), 451-559. doi: 10.1017/S0962492910000061.  Google Scholar

[1]

Junxiong Jia, Jigen Peng, Jinghuai Gao. Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption. Inverse Problems & Imaging, 2021, 15 (2) : 201-228. doi: 10.3934/ipi.2020061
[2]

Jiangfeng Huang, Zhiliang Deng, Liwei Xu. A Bayesian level set method for an inverse medium scattering problem in acoustics. Inverse Problems & Imaging, 2021, 15 (5) : 1077-1097. doi: 10.3934/ipi.2021029
[3]

Sebastian Reich, Paul J. Rozdeba. Posterior contraction rates for non-parametric state and drift estimation. Foundations of Data Science, 2020, 2 (3) : 333-349. doi: 10.3934/fods.2020016
[4]

Masoumeh Dashti, Stephen Harris, Andrew Stuart. Besov priors for Bayesian inverse problems. Inverse Problems & Imaging, 2012, 6 (2) : 183-200. doi: 10.3934/ipi.2012.6.183
[5]

Guillaume Bal, Ian Langmore, Youssef Marzouk. Bayesian inverse problems with Monte Carlo forward models. Inverse Problems & Imaging, 2013, 7 (1) : 81-105. doi: 10.3934/ipi.2013.7.81
[6]

Sari Lasanen. Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns. Inverse Problems & Imaging, 2012, 6 (2) : 267-287. doi: 10.3934/ipi.2012.6.267
[7]

Sari Lasanen. Non-Gaussian statistical inverse problems. Part I: Posterior distributions. Inverse Problems & Imaging, 2012, 6 (2) : 215-266. doi: 10.3934/ipi.2012.6.215
[8]

Tan Bui-Thanh, Omar Ghattas. A scalable algorithm for MAP estimators in Bayesian inverse problems with Besov priors. Inverse Problems & Imaging, 2015, 9 (1) : 27-53. doi: 10.3934/ipi.2015.9.27
[9]

Victor Isakov, Joseph Myers. On the inverse doping profile problem. Inverse Problems & Imaging, 2012, 6 (3) : 465-486. doi: 10.3934/ipi.2012.6.465
[10]

Tan Bui-Thanh, Quoc P. Nguyen. FEM-based discretization-invariant MCMC methods for PDE-constrained Bayesian inverse problems. Inverse Problems & Imaging, 2016, 10 (4) : 943-975. doi: 10.3934/ipi.2016028
[11]

T. J. Sullivan. Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors. Inverse Problems & Imaging, 2017, 11 (5) : 857-874. doi: 10.3934/ipi.2017040
[12]

Didi Lv, Qingping Zhou, Jae Kyu Choi, Jinglai Li, Xiaoqun Zhang. Nonlocal TV-Gaussian prior for Bayesian inverse problems with applications to limited CT reconstruction. Inverse Problems & Imaging, 2020, 14 (1) : 117-132. doi: 10.3934/ipi.2019066
[13]

Meixin Xiong, Liuhong Chen, Ju Ming, Jaemin Shin. Accelerating the Bayesian inference of inverse problems by using data-driven compressive sensing method based on proper orthogonal decomposition. Electronic Research Archive, , () : -. doi: 10.3934/era.2021044
[14]

Michael L. Frankel, Victor Roytburd. Fractal dimension of attractors for a Stefan problem. Conference Publications, 2003, 2003 (Special) : 281-287. doi: 10.3934/proc.2003.2003.281
[15]

Lincoln Chayes, Inwon C. Kim. The supercooled Stefan problem in one dimension. Communications on Pure & Applied Analysis, 2012, 11 (2) : 845-859. doi: 10.3934/cpaa.2012.11.845
[16]

Kazuhiro Kawamura. Mean dimension of shifts of finite type and of generalized inverse limits. Discrete & Continuous Dynamical Systems, 2020, 40 (8) : 4767-4775. doi: 10.3934/dcds.2020200
[17]

Alexandr Mikhaylov, Victor Mikhaylov. Dynamic inverse problem for Jacobi matrices. Inverse Problems & Imaging, 2019, 13 (3) : 431-447. doi: 10.3934/ipi.2019021
[18]

Armin Lechleiter, Tobias Rienmüller. Factorization method for the inverse Stokes problem. Inverse Problems & Imaging, 2013, 7 (4) : 1271-1293. doi: 10.3934/ipi.2013.7.1271
[19]

Nguyen Huy Tuan, Mokhtar Kirane, Long Dinh Le, Van Thinh Nguyen. On an inverse problem for fractional evolution equation. Evolution Equations & Control Theory, 2017, 6 (1) : 111-134. doi: 10.3934/eect.2017007
[20]

Russell Johnson, Luca Zampogni. On the inverse Sturm-Liouville problem. Discrete & Continuous Dynamical Systems, 2007, 18 (2&3) : 405-428. doi: 10.3934/dcds.2007.18.405

2020 Impact Factor: 1.639

Tools

Metrics

  • PDF downloads (94)
  • HTML views (0)
  • Cited by (9)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences