American Institute of Mathematical Sciences

Advanced
April  2021, 15(2): 201-228. doi: 10.3934/ipi.2020061

Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption

Junxiong Jia 1, , Jigen Peng 2,, and  Jinghuai Gao 3,

1. 

School of Mathematics and Statistics, Xi'an Jiaotong University, Xi'an, 710049, China
2. 

School of Mathematics and Information Science, Guangzhou University, Guangzhou, 510006, China
3. 

School of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an, 710049, China

* Corresponding author: Jigen Peng

Received  February 2020 Revised  August 2020 Published  April 2021 Early access  October 2020

We investigate an empirical Bayesian nonparametric approach to a family of linear inverse problems with Gaussian prior and Gaussian noise. We consider a class of Gaussian prior probability measures with covariance operator indexed by a hyperparameter that quantifies regularity. By introducing two auxiliary problems, we construct an empirical Bayes method and prove that this method can automatically select the hyperparameter. In addition, we show that this adaptive Bayes procedure provides optimal contraction rates up to a slowly varying term and an arbitrarily small constant, without knowledge about the regularity index. Our method needs not the prior covariance, noise covariance and forward operator have a common basis in their singular value decomposition, enlarging the application range compared with the existing results. A simple simulation example is given that illustrates the effectiveness of the proposed method.

Keywords: posterior consistency, empirical Bayesian approach, linear inverse problem, non-simultaneous diagonal.
Mathematics Subject Classification: Primary: 35R30, 35R25; Secondary: 45Q05.
Citation: Junxiong Jia, Jigen Peng, Jinghuai Gao. Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption. Inverse Problems & Imaging, 2021, 15 (2) : 201-228. doi: 10.3934/ipi.2020061
References:
[1]

S. Agapiou, S. Larsson and A. W. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems, Stoch. Proc. Appl., 123 (2013), 3828-3860.  doi: 10.1016/j.spa.2013.05.001.  Google Scholar

[2]

S. Agapiou and P. Mathé, Posterior contraction in bayesian inverse problems under gaussian priors, in New Trends in Parameter Identification for Mathematical Models (eds. B. Hofmann, A. Leit$\tilde{o}$ and J. P. Zubelli), 2018, 1–29. doi: 10.1007/978-3-319-70824-9_1.  Google Scholar

[3]

E. Belitser and S. Ghosal, Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution, Ann. Statist., 31 (2003), 536-559.  doi: 10.1214/aos/1051027880.  Google Scholar

[4]

T. Bui-Thanh and Q. P. Nguyen, FEM-based discretization-invariant MCMC methods for PDE-constrained Bayesian inverse problems, Inverse Probl. Imag., 10 (2016), 943-975.  doi: 10.3934/ipi.2016028.  Google Scholar

[5]

A. D. Bull, Honest adaptive confidence bands and self-similar functions, Electron. J. Stat., 6 (2012), 1490-1516.  doi: 10.1214/12-EJS720.  Google Scholar

[6]

M. Burger and F. Lucka, Maximum a posteriori estimates in linear inverse problems with log-concave priors are proper Bayes estimators, Inverse Probl., 30 (2014), 114004, 21pp. doi: 10.1088/0266-5611/30/11/114004.  Google Scholar

[7]

S. L. Cotter, M. Dashti and J. C. Robinson and A. M. Stuart, Bayesian inverse problems for functions and applications to fluid mechanics, Inverse Probl., 25 (2009), 115008, 43pp. doi: 10.1088/0266-5611/25/11/115008.  Google Scholar

[8]

S. L. Cotter, G. O. Roberts, A. M. Stuart and D. White, MCMC methods for functions: Modifying old algorithms to make them faster, Stat. Sci., 28 (2013), 424-446.  doi: 10.1214/13-STS421.  Google Scholar

[9]

M. Dashti, K. J. H. Law and A. M. Stuart and J. Voss, MAP estimators and their consistency in Bayesian nonparametric inverse problems, Inverse Probl., 29 (2013), 095017, 27pp. doi: 10.1088/0266-5611/29/9/095017.  Google Scholar

[10]

M. Dashti and A. W. Stuart, The Bayesian Approach to Inverse Problems, in Handbook of Uncertainty Quantification (eds. R. Ghanem, D. Higdon and H. Owhadi), Springer, Cham, 2017.  Google Scholar

[11]

M. M. Dunlop and A. M. Stuart, MAP estimators for piecewise continuous inversion, Inverse Probl., 32 (2016), 105003, 50pp. doi: 10.1088/0266-5611/32/10/105003.  Google Scholar

[12]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Springer, Netherlands, 1996.  Google Scholar

[13]

L. C. Evans, Partial Differential Equations, 2$^nd$ edition, American Mathematical Society, United States, 2010. doi: 10.1090/gsm/019.  Google Scholar

[14]

S. Ghosal and A. Van Der Vaart, Convergence rates of posterior distributions for noniid observations, Ann. Statist., 35 (2007), 192-223.  doi: 10.1214/009053606000001172.  Google Scholar

[15]

E. Giné and R. Nickl, Confidence bands in density estimation, Ann. Statist., 38 (2010), 1122-1170.  doi: 10.1214/09-AOS738.  Google Scholar

[16]

T. Hsing and R. Eubank, Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators, John Wiley & Sons, Ltd., Chichester, 2015. doi: 10.1002/9781118762547.  Google Scholar

[17]

M. Ilić, F. Liu, I. Turner and V. Anh, Numerical approximation of a fractional-in-space diffusion equation, Frac. Calc. App. Anal., 8 (2005), 323-341.   Google Scholar

[18]

J. Jia, J. Peng and J. Gao, Bayesian approach to inverse problems for functions with a variable-index Besov prior, Inverse Probl., 32 (2016), 085006, 32pp. doi: 10.1088/0266-5611/32/8/085006.  Google Scholar

[19]

J. Jia, S. Yue, J. Peng and J. Gao, Infinite-dimensional Bayesian approach for inverse scattering problems of a fractional Helmholtz equation, J. Funct. Anal., 275 (2016), 2299-2332.  doi: 10.1016/j.jfa.2018.08.002.  Google Scholar

[20]

H. Kekkonen, M. Lassas and S. Siltanen, Posterior consistency and convergence rates for Bayesian inversion with hypoelliptic operators, Inverse Probl., 32 (2016), 085005, 31pp. doi: 10.1088/0266-5611/32/8/085005.  Google Scholar

[21]

B. T. Knapik, A. van Der Vaart and J. H. van Zanten, Bayesian inverse problems with Gaussian priors, Ann. Statist., 39 (2011), 2626-2657.  doi: 10.1214/11-AOS920.  Google Scholar

[22]

B. T. Knapik and J. B. Salomond, A general approach to posterior contraction in nonparametric inverse problems, Bernoulli, 24 (2018), 2091-2121.  doi: 10.3150/16-BEJ921.  Google Scholar

[23]

B. T. Knapik, B. T. Szabó, A. W. van der Vaart and J. H. van Zanten, Bayes procedures for adaptive inference in inverse problems for the white noise model, Probab. Theory Rel., 164 (2016), 771-813.  doi: 10.1007/s00440-015-0619-7.  Google Scholar

[24]

J. Koponen, T. Huttunen, T. Tarvainen and J. P. Kaipio, Bayesian approximation error approach in full-wave ultrasound tomography, IEEE T. Ultrason. Ferr., 61 (2014), 1627-1637.  doi: 10.1109/TUFFC.2014.006319.  Google Scholar

[25]

P. Mathé, Bayesian inverse problems with non-commuting operators, Math. Comput., 88 (2019), 2897-2912.  doi: 10.1090/mcom/3439.  Google Scholar

[26]

G. D. Prato, An Introduction to Infinite-Dimensional Analysis, Scuola Normale Superiore, Pisa, 2001. doi: 10.1007/3-540-29021-4.  Google Scholar

[27]

M. A. Shubin and S. I. Andersson, Pseudodifferential Operators and Spectral Theory, Springer, Berlin, 2001.  Google Scholar

[28]

A. W. Stuart, Inverse problems: A Bayesian perspective, Acta Numer., 19 (2010), 451-559.  doi: 10.1017/S0962492910000061.  Google Scholar

[29]

B. Szabó, A. W. van der Vaart and J. H. van Zanten, Frequentist coverage of adaptive nonparametric Bayesian credible sets, Ann. Statist., 43 (2015), 1391-1428.  doi: 10.1214/14-AOS1270.  Google Scholar

[30]

M. Trabs, Bayesian inverse problems with unknown operators, Inverse Probl., 34 (2018), 085001, 27pp. doi: 10.1088/1361-6420/aac3aa.  Google Scholar

[31]

S. J. Vollmer, Posterior consistency for Bayesian inverse problems through stability and regression results, Inverse Probl., 29 (2013), 125011, 32pp. doi: 10.1088/0266-5611/29/12/125011.  Google Scholar

show all references

References:
[1]

S. Agapiou, S. Larsson and A. W. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems, Stoch. Proc. Appl., 123 (2013), 3828-3860.  doi: 10.1016/j.spa.2013.05.001.  Google Scholar

[2]

S. Agapiou and P. Mathé, Posterior contraction in bayesian inverse problems under gaussian priors, in New Trends in Parameter Identification for Mathematical Models (eds. B. Hofmann, A. Leit$\tilde{o}$ and J. P. Zubelli), 2018, 1–29. doi: 10.1007/978-3-319-70824-9_1.  Google Scholar

[3]

E. Belitser and S. Ghosal, Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution, Ann. Statist., 31 (2003), 536-559.  doi: 10.1214/aos/1051027880.  Google Scholar

[4]

T. Bui-Thanh and Q. P. Nguyen, FEM-based discretization-invariant MCMC methods for PDE-constrained Bayesian inverse problems, Inverse Probl. Imag., 10 (2016), 943-975.  doi: 10.3934/ipi.2016028.  Google Scholar

[5]

A. D. Bull, Honest adaptive confidence bands and self-similar functions, Electron. J. Stat., 6 (2012), 1490-1516.  doi: 10.1214/12-EJS720.  Google Scholar

[6]

M. Burger and F. Lucka, Maximum a posteriori estimates in linear inverse problems with log-concave priors are proper Bayes estimators, Inverse Probl., 30 (2014), 114004, 21pp. doi: 10.1088/0266-5611/30/11/114004.  Google Scholar

[7]

S. L. Cotter, M. Dashti and J. C. Robinson and A. M. Stuart, Bayesian inverse problems for functions and applications to fluid mechanics, Inverse Probl., 25 (2009), 115008, 43pp. doi: 10.1088/0266-5611/25/11/115008.  Google Scholar

[8]

S. L. Cotter, G. O. Roberts, A. M. Stuart and D. White, MCMC methods for functions: Modifying old algorithms to make them faster, Stat. Sci., 28 (2013), 424-446.  doi: 10.1214/13-STS421.  Google Scholar

[9]

M. Dashti, K. J. H. Law and A. M. Stuart and J. Voss, MAP estimators and their consistency in Bayesian nonparametric inverse problems, Inverse Probl., 29 (2013), 095017, 27pp. doi: 10.1088/0266-5611/29/9/095017.  Google Scholar

[10]

M. Dashti and A. W. Stuart, The Bayesian Approach to Inverse Problems, in Handbook of Uncertainty Quantification (eds. R. Ghanem, D. Higdon and H. Owhadi), Springer, Cham, 2017.  Google Scholar

[11]

M. M. Dunlop and A. M. Stuart, MAP estimators for piecewise continuous inversion, Inverse Probl., 32 (2016), 105003, 50pp. doi: 10.1088/0266-5611/32/10/105003.  Google Scholar

[12]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Springer, Netherlands, 1996.  Google Scholar

[13]

L. C. Evans, Partial Differential Equations, 2$^nd$ edition, American Mathematical Society, United States, 2010. doi: 10.1090/gsm/019.  Google Scholar

[14]

S. Ghosal and A. Van Der Vaart, Convergence rates of posterior distributions for noniid observations, Ann. Statist., 35 (2007), 192-223.  doi: 10.1214/009053606000001172.  Google Scholar

[15]

E. Giné and R. Nickl, Confidence bands in density estimation, Ann. Statist., 38 (2010), 1122-1170.  doi: 10.1214/09-AOS738.  Google Scholar

[16]

T. Hsing and R. Eubank, Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators, John Wiley & Sons, Ltd., Chichester, 2015. doi: 10.1002/9781118762547.  Google Scholar

[17]

M. Ilić, F. Liu, I. Turner and V. Anh, Numerical approximation of a fractional-in-space diffusion equation, Frac. Calc. App. Anal., 8 (2005), 323-341.   Google Scholar

[18]

J. Jia, J. Peng and J. Gao, Bayesian approach to inverse problems for functions with a variable-index Besov prior, Inverse Probl., 32 (2016), 085006, 32pp. doi: 10.1088/0266-5611/32/8/085006.  Google Scholar

[19]

J. Jia, S. Yue, J. Peng and J. Gao, Infinite-dimensional Bayesian approach for inverse scattering problems of a fractional Helmholtz equation, J. Funct. Anal., 275 (2016), 2299-2332.  doi: 10.1016/j.jfa.2018.08.002.  Google Scholar

[20]

H. Kekkonen, M. Lassas and S. Siltanen, Posterior consistency and convergence rates for Bayesian inversion with hypoelliptic operators, Inverse Probl., 32 (2016), 085005, 31pp. doi: 10.1088/0266-5611/32/8/085005.  Google Scholar

[21]

B. T. Knapik, A. van Der Vaart and J. H. van Zanten, Bayesian inverse problems with Gaussian priors, Ann. Statist., 39 (2011), 2626-2657.  doi: 10.1214/11-AOS920.  Google Scholar

[22]

B. T. Knapik and J. B. Salomond, A general approach to posterior contraction in nonparametric inverse problems, Bernoulli, 24 (2018), 2091-2121.  doi: 10.3150/16-BEJ921.  Google Scholar

[23]

B. T. Knapik, B. T. Szabó, A. W. van der Vaart and J. H. van Zanten, Bayes procedures for adaptive inference in inverse problems for the white noise model, Probab. Theory Rel., 164 (2016), 771-813.  doi: 10.1007/s00440-015-0619-7.  Google Scholar

[24]

J. Koponen, T. Huttunen, T. Tarvainen and J. P. Kaipio, Bayesian approximation error approach in full-wave ultrasound tomography, IEEE T. Ultrason. Ferr., 61 (2014), 1627-1637.  doi: 10.1109/TUFFC.2014.006319.  Google Scholar

[25]

P. Mathé, Bayesian inverse problems with non-commuting operators, Math. Comput., 88 (2019), 2897-2912.  doi: 10.1090/mcom/3439.  Google Scholar

[26]

G. D. Prato, An Introduction to Infinite-Dimensional Analysis, Scuola Normale Superiore, Pisa, 2001. doi: 10.1007/3-540-29021-4.  Google Scholar

[27]

M. A. Shubin and S. I. Andersson, Pseudodifferential Operators and Spectral Theory, Springer, Berlin, 2001.  Google Scholar

[28]

A. W. Stuart, Inverse problems: A Bayesian perspective, Acta Numer., 19 (2010), 451-559.  doi: 10.1017/S0962492910000061.  Google Scholar

[29]

B. Szabó, A. W. van der Vaart and J. H. van Zanten, Frequentist coverage of adaptive nonparametric Bayesian credible sets, Ann. Statist., 43 (2015), 1391-1428.  doi: 10.1214/14-AOS1270.  Google Scholar

[30]

M. Trabs, Bayesian inverse problems with unknown operators, Inverse Probl., 34 (2018), 085001, 27pp. doi: 10.1088/1361-6420/aac3aa.  Google Scholar

[31]

S. J. Vollmer, Posterior consistency for Bayesian inverse problems through stability and regression results, Inverse Probl., 29 (2013), 125011, 32pp. doi: 10.1088/0266-5611/29/12/125011.  Google Scholar

Figure 1.  Left panels the empirical Bayes posterior mean (red) and the true curve (blue, dashed). Right panels corresponding normalized likelihood for $ \hat{\tilde{\alpha}} $ (regularity index for the artificial diagonal problem). We have $ n = 10^3, 10^5, 10^8, $ and $ 10^{12} $, from top to bottom
Figure Options

Download full-size image

Download as PowerPoint slide

[1]

Kui Lin, Shuai Lu, Peter Mathé. Oracle-type posterior contraction rates in Bayesian inverse problems. Inverse Problems & Imaging, 2015, 9 (3) : 895-915. doi: 10.3934/ipi.2015.9.895
[2]

C. Brändle, F. Quirós, Julio D. Rossi. Non-simultaneous blow-up for a quasilinear parabolic system with reaction at the boundary. Communications on Pure & Applied Analysis, 2005, 4 (3) : 523-536. doi: 10.3934/cpaa.2005.4.523
[3]

Lan Qiao, Sining Zheng. Non-simultaneous blow-up for heat equations with positive-negative sources and coupled boundary flux. Communications on Pure & Applied Analysis, 2007, 6 (4) : 1113-1129. doi: 10.3934/cpaa.2007.6.1113
[4]

Sari Lasanen. Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns. Inverse Problems & Imaging, 2012, 6 (2) : 267-287. doi: 10.3934/ipi.2012.6.267
[5]

Sari Lasanen. Non-Gaussian statistical inverse problems. Part I: Posterior distributions. Inverse Problems & Imaging, 2012, 6 (2) : 215-266. doi: 10.3934/ipi.2012.6.215
[6]

Jiangfeng Huang, Zhiliang Deng, Liwei Xu. A Bayesian level set method for an inverse medium scattering problem in acoustics. Inverse Problems & Imaging, 2021, 15 (5) : 1077-1097. doi: 10.3934/ipi.2021029
[7]

Mikko Orispää, Markku Lehtinen. Fortran linear inverse problem solver. Inverse Problems & Imaging, 2010, 4 (3) : 485-503. doi: 10.3934/ipi.2010.4.485
[8]

Masoumeh Dashti, Stephen Harris, Andrew Stuart. Besov priors for Bayesian inverse problems. Inverse Problems & Imaging, 2012, 6 (2) : 183-200. doi: 10.3934/ipi.2012.6.183
[9]

Bariş Keçeci, Fulya Altıparmak, İmdat Kara. A mathematical formulation and heuristic approach for the heterogeneous fixed fleet vehicle routing problem with simultaneous pickup and delivery. Journal of Industrial & Management Optimization, 2021, 17 (3) : 1069-1100. doi: 10.3934/jimo.2020012
[10]

Yi Zhang, Yong Jiang, Liwei Zhang, Jiangzhong Zhang. A perturbation approach for an inverse linear second-order cone programming. Journal of Industrial & Management Optimization, 2013, 9 (1) : 171-189. doi: 10.3934/jimo.2013.9.171
[11]

Shiyun Wang, Yong-Jin Liu, Yong Jiang. A majorized penalty approach to inverse linear second order cone programming problems. Journal of Industrial & Management Optimization, 2014, 10 (3) : 965-976. doi: 10.3934/jimo.2014.10.965
[12]

Yibing Lv, Zhongping Wan. Linear bilevel multiobjective optimization problem: Penalty approach. Journal of Industrial & Management Optimization, 2019, 15 (3) : 1213-1223. doi: 10.3934/jimo.2018092
[13]

William Chad Young, Adrian E. Raftery, Ka Yee Yeung. A posterior probability approach for gene regulatory network inference in genetic perturbation data. Mathematical Biosciences & Engineering, 2016, 13 (6) : 1241-1251. doi: 10.3934/mbe.2016041
[14]

Rakesh Pilkar, Erik M. Bollt, Charles Robinson. Empirical mode decomposition/Hilbert transform analysis of postural responses to small amplitude anterior-posterior sinusoidal translations of varying frequencies. Mathematical Biosciences & Engineering, 2011, 8 (4) : 1085-1097. doi: 10.3934/mbe.2011.8.1085
[15]

Ben Schweizer, Marco Veneroni. The needle problem approach to non-periodic homogenization. Networks & Heterogeneous Media, 2011, 6 (4) : 755-781. doi: 10.3934/nhm.2011.6.755
[16]

Lili Chang, Wei Gong, Guiquan Sun, Ningning Yan. PDE-constrained optimal control approach for the approximation of an inverse Cauchy problem. Inverse Problems & Imaging, 2015, 9 (3) : 791-814. doi: 10.3934/ipi.2015.9.791
[17]

Junfeng Yang. Dynamic power price problem: An inverse variational inequality approach. Journal of Industrial & Management Optimization, 2008, 4 (4) : 673-684. doi: 10.3934/jimo.2008.4.673
[18]

Laurent Bourgeois, Jérémi Dardé. A quasi-reversibility approach to solve the inverse obstacle problem. Inverse Problems & Imaging, 2010, 4 (3) : 351-377. doi: 10.3934/ipi.2010.4.351
[19]

Laurent Bourgeois, Jérémi Dardé. The "exterior approach" to solve the inverse obstacle problem for the Stokes system. Inverse Problems & Imaging, 2014, 8 (1) : 23-51. doi: 10.3934/ipi.2014.8.23
[20]

Guillaume Bal, Ian Langmore, Youssef Marzouk. Bayesian inverse problems with Monte Carlo forward models. Inverse Problems & Imaging, 2013, 7 (1) : 81-105. doi: 10.3934/ipi.2013.7.81

2020 Impact Factor: 1.639

Tools

Metrics

  • PDF downloads (182)
  • HTML views (210)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences