Oracle-type posterior contraction rates in Bayesian inverse problems

Previous Article
A reweighted $l^2$ method for image restoration with Poisson and mixed Poisson-Gaussian noise
IPI Home
This Issue
Next Article
Hyperspectral unmixing by the alternating direction method of multipliers

August 2015, 9(3): 895-915. doi: 10.3934/ipi.2015.9.895

Oracle-type posterior contraction rates in Bayesian inverse problems

Kui Lin ^1, , Shuai Lu ^1, and Peter Mathé ^2,

1.	School of Mathematical Sciences, Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University, Shanghai, 200433, China, China
2.	Weierstraß Institute for Applied Analysis and Stochastics, Mohrenstraße 39, 10117 Berlin, Germany

Received April 2014 Revised June 2015 Published July 2015

Abstract
Related Papers

We discuss Bayesian inverse problems in Hilbert spaces. The focus is on a fast concentration of the posterior probability around the unknown true solution as expressed in the concept of posterior contraction rates. This concentration is dominated by a parameter which controls the variance of the prior distribution. Previous results determine posterior contraction rates based on known solution smoothness. Here we show that an oracle-type parameter choice is possible. This is done by relating the posterior contraction rate to the root mean squared estimation error. In addition we show that the tail probability, which usually is bounded by using the Chebyshev inequality, has exponential decay, at least for a priori parameter choices. These results implement the exponential concentration of Gaussian measures in Hilbert spaces.

Keywords: effective dimension., posterior contraction, Bayesian inverse problem.

Mathematics Subject Classification: Primary: 62F15; Secondary: 65J20, 65J22, 62C1.

Citation: Kui Lin, Shuai Lu, Peter Mathé. Oracle-type posterior contraction rates in Bayesian inverse problems. Inverse Problems & Imaging, 2015, 9 (3) : 895-915. doi: 10.3934/ipi.2015.9.895

References:

[1]	S. Agapiou, Aspects of Bayesian Inverse Problems,, Ph.D. thesis, (2013).
[2]	S. Agapiou, S. Larsson and A. M. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems,, Stochastic Process. Appl., 123 (2013), 3828. doi: 10.1016/j.spa.2013.05.001.
[3]	S. Agapiou and P. Mathé, Preconditioning the prior to avoid saturation in Bayesian inverse problems,, submitted, (2014).
[4]	F. Bauer and M. A. Lukas, Comparing parameter choice methods for regularization of ill-posed problems,, Math. Comput. Simulation, 81 (2011), 1795. doi: 10.1016/j.matcom.2011.01.016.
[5]	N. Bissantz, T. Hohage and A. Munk, Consistency and rates of convergence of nonlinear Tikhonov regularization with random noise,, Inverse Problems, 20 (2004), 1773. doi: 10.1088/0266-5611/20/6/005.
[6]	G. Blanchard and P. Mathé, Discrepancy principle for statistical inverse problems with application to conjugate gradient regularization,, Inverse Problems, 28 (2012). doi: 10.1088/0266-5611/28/11/115011.
[7]	A. Caponnetto and E. De Vito, Optimal rates for the regularized least-squares algorithm,, Found. Comput. Math., 7 (2007), 331. doi: 10.1007/s10208-006-0196-8.
[8]	H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems,, Mathematics and its Applications, (1996). doi: 10.1007/978-94-009-1740-8.
[9]	B. G. Fitzpatrick, Bayesian analysis in inverse problems,, Inverse Problems, 7 (1991), 675. doi: 10.1088/0266-5611/7/5/003.
[10]	J.-P. Florens and A. Simoni, Regularizing priors for linear inverse problems,, Scand. J. Stat., 39 (2012), 214. doi: 10.1111/j.1467-9469.2011.00784.x.
[11]	P. C. Hansen, Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems,, Numer. Algorithms, 6 (1994), 1. doi: 10.1007/BF02149761.
[12]	J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems,, Applied Mathematical Sciences, (2005).
[13]	B. T. Knapik, A. W. van der Vaart and H. van Zanten, Bayesian inverse problems with Gaussian priors,, Ann. Statist., 39 (2011), 2626. doi: 10.1214/11-AOS920.
[14]	S. Lasanen, Non-Gaussian statistical inverse problems. Part I: Posterior distributions,, Inverse Probl. Imaging, 6 (2012), 215. doi: 10.3934/ipi.2012.6.215.
[15]	S. Lasanen, Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns,, Inverse Probl. Imaging, 6 (2012), 267. doi: 10.3934/ipi.2012.6.267.
[16]	M. Lassas, E. Saksman and S. Siltanen, Discretization-invariant Bayesian inversion and Besov space priors,, Inverse Problems and Imaging, 3 (2009), 87. doi: 10.3934/ipi.2009.3.87.
[17]	M. Lassas and S. Siltanen, Can one use total variation prior for edge-preserving Bayesian inversion?,, Inverse Problems, 20 (2004), 1537. doi: 10.1088/0266-5611/20/5/013.
[18]	M. Ledoux and M. Talagrand, Probability in Banach Spaces. Isoperimetry and Processes,, Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)], (1991). doi: 10.1007/978-3-642-20212-4.
[19]	M. S. Lehtinen, L. Päivärinta and E. Somersalo, Linear inverse problems for generalised random variables,, Inverse Problems, 5 (1989), 599. doi: 10.1088/0266-5611/5/4/011.
[20]	S. Lu and S. V. Pereverzev, Regularization Theory for Ill-Posed Problems. SElected Topics,, Inverse and Ill-posed Problems Series, (2013). doi: 10.1515/9783110286496.
[21]	S. Lu and P. Mathé, Discrepancy based model selection in statistical inverse problems,, J. Complexity, 30 (2014), 290. doi: 10.1016/j.jco.2014.02.002.
[22]	A. Mandelbaum, Linear estimators and measurable linear transformations on a Hilbert space,, Z. Wahrsch. Verw. Gebiete, 65 (1984), 385. doi: 10.1007/BF00533743.
[23]	C. Marteau and P. Mathé, General regularization schemes for signal detection in inverse problems,, Math. Meth. Statist., 23 (2014), 176. doi: 10.3103/S1066530714030028.
[24]	P. Mathé and B. Hofmann, How general are general source conditions?,, Inverse Problems, 24 (2008). doi: 10.1088/0266-5611/24/1/015009.
[25]	P. Mathé and S. V. Pereverzev, Geometry of linear ill-posed problems in variable Hilbert scales,, Inverse Problems, 19 (2003), 789. doi: 10.1088/0266-5611/19/3/319.
[26]	P. Mathé and S. V. Pereverzev, Regularization of some linear ill-posed problems with discretized random noisy data,, Math. Comp., 75 (2006), 1913. doi: 10.1090/S0025-5718-06-01873-4.
[27]	P. Mathé and U. Tautenhahn, Enhancing linear regularization to treat large noise,, J. Inverse Ill-Posed Probl., 19 (2011), 859. doi: 10.1515/JIIP.2011.052.
[28]	A. M. Stuart, Inverse problems: A Bayesian perspective,, Acta Numer., 19 (2010), 451. doi: 10.1017/S0962492910000061.

show all references

References:

[1]	S. Agapiou, Aspects of Bayesian Inverse Problems,, Ph.D. thesis, (2013).
[2]	S. Agapiou, S. Larsson and A. M. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems,, Stochastic Process. Appl., 123 (2013), 3828. doi: 10.1016/j.spa.2013.05.001.
[3]	S. Agapiou and P. Mathé, Preconditioning the prior to avoid saturation in Bayesian inverse problems,, submitted, (2014).
[4]	F. Bauer and M. A. Lukas, Comparing parameter choice methods for regularization of ill-posed problems,, Math. Comput. Simulation, 81 (2011), 1795. doi: 10.1016/j.matcom.2011.01.016.
[5]	N. Bissantz, T. Hohage and A. Munk, Consistency and rates of convergence of nonlinear Tikhonov regularization with random noise,, Inverse Problems, 20 (2004), 1773. doi: 10.1088/0266-5611/20/6/005.
[6]	G. Blanchard and P. Mathé, Discrepancy principle for statistical inverse problems with application to conjugate gradient regularization,, Inverse Problems, 28 (2012). doi: 10.1088/0266-5611/28/11/115011.
[7]	A. Caponnetto and E. De Vito, Optimal rates for the regularized least-squares algorithm,, Found. Comput. Math., 7 (2007), 331. doi: 10.1007/s10208-006-0196-8.
[8]	H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems,, Mathematics and its Applications, (1996). doi: 10.1007/978-94-009-1740-8.
[9]	B. G. Fitzpatrick, Bayesian analysis in inverse problems,, Inverse Problems, 7 (1991), 675. doi: 10.1088/0266-5611/7/5/003.
[10]	J.-P. Florens and A. Simoni, Regularizing priors for linear inverse problems,, Scand. J. Stat., 39 (2012), 214. doi: 10.1111/j.1467-9469.2011.00784.x.
[11]	P. C. Hansen, Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems,, Numer. Algorithms, 6 (1994), 1. doi: 10.1007/BF02149761.
[12]	J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems,, Applied Mathematical Sciences, (2005).
[13]	B. T. Knapik, A. W. van der Vaart and H. van Zanten, Bayesian inverse problems with Gaussian priors,, Ann. Statist., 39 (2011), 2626. doi: 10.1214/11-AOS920.
[14]	S. Lasanen, Non-Gaussian statistical inverse problems. Part I: Posterior distributions,, Inverse Probl. Imaging, 6 (2012), 215. doi: 10.3934/ipi.2012.6.215.
[15]	S. Lasanen, Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns,, Inverse Probl. Imaging, 6 (2012), 267. doi: 10.3934/ipi.2012.6.267.
[16]	M. Lassas, E. Saksman and S. Siltanen, Discretization-invariant Bayesian inversion and Besov space priors,, Inverse Problems and Imaging, 3 (2009), 87. doi: 10.3934/ipi.2009.3.87.
[17]	M. Lassas and S. Siltanen, Can one use total variation prior for edge-preserving Bayesian inversion?,, Inverse Problems, 20 (2004), 1537. doi: 10.1088/0266-5611/20/5/013.
[18]	M. Ledoux and M. Talagrand, Probability in Banach Spaces. Isoperimetry and Processes,, Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)], (1991). doi: 10.1007/978-3-642-20212-4.
[19]	M. S. Lehtinen, L. Päivärinta and E. Somersalo, Linear inverse problems for generalised random variables,, Inverse Problems, 5 (1989), 599. doi: 10.1088/0266-5611/5/4/011.
[20]	S. Lu and S. V. Pereverzev, Regularization Theory for Ill-Posed Problems. SElected Topics,, Inverse and Ill-posed Problems Series, (2013). doi: 10.1515/9783110286496.
[21]	S. Lu and P. Mathé, Discrepancy based model selection in statistical inverse problems,, J. Complexity, 30 (2014), 290. doi: 10.1016/j.jco.2014.02.002.
[22]	A. Mandelbaum, Linear estimators and measurable linear transformations on a Hilbert space,, Z. Wahrsch. Verw. Gebiete, 65 (1984), 385. doi: 10.1007/BF00533743.
[23]	C. Marteau and P. Mathé, General regularization schemes for signal detection in inverse problems,, Math. Meth. Statist., 23 (2014), 176. doi: 10.3103/S1066530714030028.
[24]	P. Mathé and B. Hofmann, How general are general source conditions?,, Inverse Problems, 24 (2008). doi: 10.1088/0266-5611/24/1/015009.
[25]	P. Mathé and S. V. Pereverzev, Geometry of linear ill-posed problems in variable Hilbert scales,, Inverse Problems, 19 (2003), 789. doi: 10.1088/0266-5611/19/3/319.
[26]	P. Mathé and S. V. Pereverzev, Regularization of some linear ill-posed problems with discretized random noisy data,, Math. Comp., 75 (2006), 1913. doi: 10.1090/S0025-5718-06-01873-4.
[27]	P. Mathé and U. Tautenhahn, Enhancing linear regularization to treat large noise,, J. Inverse Ill-Posed Probl., 19 (2011), 859. doi: 10.1515/JIIP.2011.052.
[28]	A. M. Stuart, Inverse problems: A Bayesian perspective,, Acta Numer., 19 (2010), 451. doi: 10.1017/S0962492910000061.

[1]	Masoumeh Dashti, Stephen Harris, Andrew Stuart. Besov priors for Bayesian inverse problems. Inverse Problems & Imaging, 2012, 6 (2) : 183-200. doi: 10.3934/ipi.2012.6.183
[2]	Guillaume Bal, Ian Langmore, Youssef Marzouk. Bayesian inverse problems with Monte Carlo forward models. Inverse Problems & Imaging, 2013, 7 (1) : 81-105. doi: 10.3934/ipi.2013.7.81
[3]	Tan Bui-Thanh, Omar Ghattas. A scalable algorithm for MAP estimators in Bayesian inverse problems with Besov priors. Inverse Problems & Imaging, 2015, 9 (1) : 27-53. doi: 10.3934/ipi.2015.9.27
[4]	Sari Lasanen. Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns. Inverse Problems & Imaging, 2012, 6 (2) : 267-287. doi: 10.3934/ipi.2012.6.267
[5]	Sari Lasanen. Non-Gaussian statistical inverse problems. Part I: Posterior distributions. Inverse Problems & Imaging, 2012, 6 (2) : 215-266. doi: 10.3934/ipi.2012.6.215
[6]	T. J. Sullivan. Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors. Inverse Problems & Imaging, 2017, 11 (5) : 857-874. doi: 10.3934/ipi.2017040
[7]	Tan Bui-Thanh, Quoc P. Nguyen. FEM-based discretization-invariant MCMC methods for PDE-constrained Bayesian inverse problems. Inverse Problems & Imaging, 2016, 10 (4) : 943-975. doi: 10.3934/ipi.2016028
[8]	Victor Isakov, Joseph Myers. On the inverse doping profile problem. Inverse Problems & Imaging, 2012, 6 (3) : 465-486. doi: 10.3934/ipi.2012.6.465
[9]	Michael L. Frankel, Victor Roytburd. Fractal dimension of attractors for a Stefan problem. Conference Publications, 2003, 2003 (Special) : 281-287. doi: 10.3934/proc.2003.2003.281
[10]	Lincoln Chayes, Inwon C. Kim. The supercooled Stefan problem in one dimension. Communications on Pure & Applied Analysis, 2012, 11 (2) : 845-859. doi: 10.3934/cpaa.2012.11.845
[11]	Armin Lechleiter, Tobias Rienmüller. Factorization method for the inverse Stokes problem. Inverse Problems & Imaging, 2013, 7 (4) : 1271-1293. doi: 10.3934/ipi.2013.7.1271
[12]	Nguyen Huy Tuan, Mokhtar Kirane, Long Dinh Le, Van Thinh Nguyen. On an inverse problem for fractional evolution equation. Evolution Equations & Control Theory, 2017, 6 (1) : 111-134. doi: 10.3934/eect.2017007
[13]	Russell Johnson, Luca Zampogni. On the inverse Sturm-Liouville problem. Discrete & Continuous Dynamical Systems - A, 2007, 18 (2&3) : 405-428. doi: 10.3934/dcds.2007.18.405
[14]	Mikko Orispää, Markku Lehtinen. Fortran linear inverse problem solver. Inverse Problems & Imaging, 2010, 4 (3) : 485-503. doi: 10.3934/ipi.2010.4.485
[15]	A. Doubov, Enrique Fernández-Cara, Manuel González-Burgos, J. H. Ortega. A geometric inverse problem for the Boussinesq system. Discrete & Continuous Dynamical Systems - B, 2006, 6 (6) : 1213-1238. doi: 10.3934/dcdsb.2006.6.1213
[16]	Alexandr Mikhaylov, Victor Mikhaylov. Dynamic inverse problem for Jacobi matrices. Inverse Problems & Imaging, 2019, 13 (3) : 431-447. doi: 10.3934/ipi.2019021
[17]	Hermann Gross, Sebastian Heidenreich, Mark-Alexander Henn, Markus Bär, Andreas Rathsfeld. Modeling aspects to improve the solution of the inverse problem in scatterometry. Discrete & Continuous Dynamical Systems - S, 2015, 8 (3) : 497-519. doi: 10.3934/dcdss.2015.8.497
[18]	Pedro Caro. On an inverse problem in electromagnetism with local data: stability and uniqueness. Inverse Problems & Imaging, 2011, 5 (2) : 297-322. doi: 10.3934/ipi.2011.5.297
[19]	Peter Monk, Virginia Selgas. Sampling type methods for an inverse waveguide problem. Inverse Problems & Imaging, 2012, 6 (4) : 709-747. doi: 10.3934/ipi.2012.6.709
[20]	Michele Di Cristo. Stability estimates in the inverse transmission scattering problem. Inverse Problems & Imaging, 2009, 3 (4) : 551-565. doi: 10.3934/ipi.2009.3.551

2017 Impact Factor: 1.465

American Institute of Mathematical Sciences