Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption

Junxiong Jia; Jigen Peng; Jinghuai Gao

doi:10.3934/ipi.2020061

Previous Article
Two new non-negativity preserving iterative regularization methods for ill-posed inverse problems
IPI Home
This Issue
Next Article
Leaf Peeling method for the wave equation on metric tree graphs

April 2021, 15(2): 201-228. doi: 10.3934/ipi.2020061

Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption

Junxiong Jia ^1, , Jigen Peng ^2,, and Jinghuai Gao ^3,

1.	School of Mathematics and Statistics, Xi'an Jiaotong University, Xi'an, 710049, China
2.	School of Mathematics and Information Science, Guangzhou University, Guangzhou, 510006, China
3.	School of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an, 710049, China

^* Corresponding author: Jigen Peng

Received February 2020 Revised August 2020 Published April 2021 Early access October 2020

We investigate an empirical Bayesian nonparametric approach to a family of linear inverse problems with Gaussian prior and Gaussian noise. We consider a class of Gaussian prior probability measures with covariance operator indexed by a hyperparameter that quantifies regularity. By introducing two auxiliary problems, we construct an empirical Bayes method and prove that this method can automatically select the hyperparameter. In addition, we show that this adaptive Bayes procedure provides optimal contraction rates up to a slowly varying term and an arbitrarily small constant, without knowledge about the regularity index. Our method needs not the prior covariance, noise covariance and forward operator have a common basis in their singular value decomposition, enlarging the application range compared with the existing results. A simple simulation example is given that illustrates the effectiveness of the proposed method.

Keywords: posterior consistency, empirical Bayesian approach, linear inverse problem, non-simultaneous diagonal.

Mathematics Subject Classification: Primary: 35R30, 35R25; Secondary: 45Q05.

Citation: Junxiong Jia, Jigen Peng, Jinghuai Gao. Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption. Inverse Problems and Imaging, 2021, 15 (2) : 201-228. doi: 10.3934/ipi.2020061

References:

[1]	S. Agapiou, S. Larsson and A. W. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems, Stoch. Proc. Appl., 123 (2013), 3828-3860. doi: 10.1016/j.spa.2013.05.001.
[2]	S. Agapiou and P. Mathé, Posterior contraction in bayesian inverse problems under gaussian priors, in New Trends in Parameter Identification for Mathematical Models (eds. B. Hofmann, A. Leit$\tilde{o}$ and J. P. Zubelli), 2018, 1–29. doi: 10.1007/978-3-319-70824-9_1.
[3]	E. Belitser and S. Ghosal, Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution, Ann. Statist., 31 (2003), 536-559. doi: 10.1214/aos/1051027880.
[4]	T. Bui-Thanh and Q. P. Nguyen, FEM-based discretization-invariant MCMC methods for PDE-constrained Bayesian inverse problems, Inverse Probl. Imag., 10 (2016), 943-975. doi: 10.3934/ipi.2016028.
[5]	A. D. Bull, Honest adaptive confidence bands and self-similar functions, Electron. J. Stat., 6 (2012), 1490-1516. doi: 10.1214/12-EJS720.
[6]	M. Burger and F. Lucka, Maximum a posteriori estimates in linear inverse problems with log-concave priors are proper Bayes estimators, Inverse Probl., 30 (2014), 114004, 21pp. doi: 10.1088/0266-5611/30/11/114004.
[7]	S. L. Cotter, M. Dashti and J. C. Robinson and A. M. Stuart, Bayesian inverse problems for functions and applications to fluid mechanics, Inverse Probl., 25 (2009), 115008, 43pp. doi: 10.1088/0266-5611/25/11/115008.
[8]	S. L. Cotter, G. O. Roberts, A. M. Stuart and D. White, MCMC methods for functions: Modifying old algorithms to make them faster, Stat. Sci., 28 (2013), 424-446. doi: 10.1214/13-STS421.
[9]	M. Dashti, K. J. H. Law and A. M. Stuart and J. Voss, MAP estimators and their consistency in Bayesian nonparametric inverse problems, Inverse Probl., 29 (2013), 095017, 27pp. doi: 10.1088/0266-5611/29/9/095017.
[10]	M. Dashti and A. W. Stuart, The Bayesian Approach to Inverse Problems, in Handbook of Uncertainty Quantification (eds. R. Ghanem, D. Higdon and H. Owhadi), Springer, Cham, 2017.
[11]	M. M. Dunlop and A. M. Stuart, MAP estimators for piecewise continuous inversion, Inverse Probl., 32 (2016), 105003, 50pp. doi: 10.1088/0266-5611/32/10/105003.
[12]	H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Springer, Netherlands, 1996.
[13]	L. C. Evans, Partial Differential Equations, 2$^nd$ edition, American Mathematical Society, United States, 2010. doi: 10.1090/gsm/019.
[14]	S. Ghosal and A. Van Der Vaart, Convergence rates of posterior distributions for noniid observations, Ann. Statist., 35 (2007), 192-223. doi: 10.1214/009053606000001172.
[15]	E. Giné and R. Nickl, Confidence bands in density estimation, Ann. Statist., 38 (2010), 1122-1170. doi: 10.1214/09-AOS738.
[16]	T. Hsing and R. Eubank, Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators, John Wiley & Sons, Ltd., Chichester, 2015. doi: 10.1002/9781118762547.
[17]	M. Ilić, F. Liu, I. Turner and V. Anh, Numerical approximation of a fractional-in-space diffusion equation, Frac. Calc. App. Anal., 8 (2005), 323-341.
[18]	J. Jia, J. Peng and J. Gao, Bayesian approach to inverse problems for functions with a variable-index Besov prior, Inverse Probl., 32 (2016), 085006, 32pp. doi: 10.1088/0266-5611/32/8/085006.
[19]	J. Jia, S. Yue, J. Peng and J. Gao, Infinite-dimensional Bayesian approach for inverse scattering problems of a fractional Helmholtz equation, J. Funct. Anal., 275 (2016), 2299-2332. doi: 10.1016/j.jfa.2018.08.002.
[20]	H. Kekkonen, M. Lassas and S. Siltanen, Posterior consistency and convergence rates for Bayesian inversion with hypoelliptic operators, Inverse Probl., 32 (2016), 085005, 31pp. doi: 10.1088/0266-5611/32/8/085005.
[21]	B. T. Knapik, A. van Der Vaart and J. H. van Zanten, Bayesian inverse problems with Gaussian priors, Ann. Statist., 39 (2011), 2626-2657. doi: 10.1214/11-AOS920.
[22]	B. T. Knapik and J. B. Salomond, A general approach to posterior contraction in nonparametric inverse problems, Bernoulli, 24 (2018), 2091-2121. doi: 10.3150/16-BEJ921.
[23]	B. T. Knapik, B. T. Szabó, A. W. van der Vaart and J. H. van Zanten, Bayes procedures for adaptive inference in inverse problems for the white noise model, Probab. Theory Rel., 164 (2016), 771-813. doi: 10.1007/s00440-015-0619-7.
[24]	J. Koponen, T. Huttunen, T. Tarvainen and J. P. Kaipio, Bayesian approximation error approach in full-wave ultrasound tomography, IEEE T. Ultrason. Ferr., 61 (2014), 1627-1637. doi: 10.1109/TUFFC.2014.006319.
[25]	P. Mathé, Bayesian inverse problems with non-commuting operators, Math. Comput., 88 (2019), 2897-2912. doi: 10.1090/mcom/3439.
[26]	G. D. Prato, An Introduction to Infinite-Dimensional Analysis, Scuola Normale Superiore, Pisa, 2001. doi: 10.1007/3-540-29021-4.
[27]	M. A. Shubin and S. I. Andersson, Pseudodifferential Operators and Spectral Theory, Springer, Berlin, 2001.
[28]	A. W. Stuart, Inverse problems: A Bayesian perspective, Acta Numer., 19 (2010), 451-559. doi: 10.1017/S0962492910000061.
[29]	B. Szabó, A. W. van der Vaart and J. H. van Zanten, Frequentist coverage of adaptive nonparametric Bayesian credible sets, Ann. Statist., 43 (2015), 1391-1428. doi: 10.1214/14-AOS1270.
[30]	M. Trabs, Bayesian inverse problems with unknown operators, Inverse Probl., 34 (2018), 085001, 27pp. doi: 10.1088/1361-6420/aac3aa.
[31]	S. J. Vollmer, Posterior consistency for Bayesian inverse problems through stability and regression results, Inverse Probl., 29 (2013), 125011, 32pp. doi: 10.1088/0266-5611/29/12/125011.

show all references

References:

[1]	S. Agapiou, S. Larsson and A. W. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems, Stoch. Proc. Appl., 123 (2013), 3828-3860. doi: 10.1016/j.spa.2013.05.001.
[2]	S. Agapiou and P. Mathé, Posterior contraction in bayesian inverse problems under gaussian priors, in New Trends in Parameter Identification for Mathematical Models (eds. B. Hofmann, A. Leit$\tilde{o}$ and J. P. Zubelli), 2018, 1–29. doi: 10.1007/978-3-319-70824-9_1.
[3]	E. Belitser and S. Ghosal, Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution, Ann. Statist., 31 (2003), 536-559. doi: 10.1214/aos/1051027880.
[4]	T. Bui-Thanh and Q. P. Nguyen, FEM-based discretization-invariant MCMC methods for PDE-constrained Bayesian inverse problems, Inverse Probl. Imag., 10 (2016), 943-975. doi: 10.3934/ipi.2016028.
[5]	A. D. Bull, Honest adaptive confidence bands and self-similar functions, Electron. J. Stat., 6 (2012), 1490-1516. doi: 10.1214/12-EJS720.
[6]	M. Burger and F. Lucka, Maximum a posteriori estimates in linear inverse problems with log-concave priors are proper Bayes estimators, Inverse Probl., 30 (2014), 114004, 21pp. doi: 10.1088/0266-5611/30/11/114004.
[7]	S. L. Cotter, M. Dashti and J. C. Robinson and A. M. Stuart, Bayesian inverse problems for functions and applications to fluid mechanics, Inverse Probl., 25 (2009), 115008, 43pp. doi: 10.1088/0266-5611/25/11/115008.
[8]	S. L. Cotter, G. O. Roberts, A. M. Stuart and D. White, MCMC methods for functions: Modifying old algorithms to make them faster, Stat. Sci., 28 (2013), 424-446. doi: 10.1214/13-STS421.
[9]	M. Dashti, K. J. H. Law and A. M. Stuart and J. Voss, MAP estimators and their consistency in Bayesian nonparametric inverse problems, Inverse Probl., 29 (2013), 095017, 27pp. doi: 10.1088/0266-5611/29/9/095017.
[10]	M. Dashti and A. W. Stuart, The Bayesian Approach to Inverse Problems, in Handbook of Uncertainty Quantification (eds. R. Ghanem, D. Higdon and H. Owhadi), Springer, Cham, 2017.
[11]	M. M. Dunlop and A. M. Stuart, MAP estimators for piecewise continuous inversion, Inverse Probl., 32 (2016), 105003, 50pp. doi: 10.1088/0266-5611/32/10/105003.
[12]	H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Springer, Netherlands, 1996.
[13]	L. C. Evans, Partial Differential Equations, 2$^nd$ edition, American Mathematical Society, United States, 2010. doi: 10.1090/gsm/019.
[14]	S. Ghosal and A. Van Der Vaart, Convergence rates of posterior distributions for noniid observations, Ann. Statist., 35 (2007), 192-223. doi: 10.1214/009053606000001172.
[15]	E. Giné and R. Nickl, Confidence bands in density estimation, Ann. Statist., 38 (2010), 1122-1170. doi: 10.1214/09-AOS738.
[16]	T. Hsing and R. Eubank, Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators, John Wiley & Sons, Ltd., Chichester, 2015. doi: 10.1002/9781118762547.
[17]	M. Ilić, F. Liu, I. Turner and V. Anh, Numerical approximation of a fractional-in-space diffusion equation, Frac. Calc. App. Anal., 8 (2005), 323-341.
[18]	J. Jia, J. Peng and J. Gao, Bayesian approach to inverse problems for functions with a variable-index Besov prior, Inverse Probl., 32 (2016), 085006, 32pp. doi: 10.1088/0266-5611/32/8/085006.
[19]	J. Jia, S. Yue, J. Peng and J. Gao, Infinite-dimensional Bayesian approach for inverse scattering problems of a fractional Helmholtz equation, J. Funct. Anal., 275 (2016), 2299-2332. doi: 10.1016/j.jfa.2018.08.002.
[20]	H. Kekkonen, M. Lassas and S. Siltanen, Posterior consistency and convergence rates for Bayesian inversion with hypoelliptic operators, Inverse Probl., 32 (2016), 085005, 31pp. doi: 10.1088/0266-5611/32/8/085005.
[21]	B. T. Knapik, A. van Der Vaart and J. H. van Zanten, Bayesian inverse problems with Gaussian priors, Ann. Statist., 39 (2011), 2626-2657. doi: 10.1214/11-AOS920.
[22]	B. T. Knapik and J. B. Salomond, A general approach to posterior contraction in nonparametric inverse problems, Bernoulli, 24 (2018), 2091-2121. doi: 10.3150/16-BEJ921.
[23]	B. T. Knapik, B. T. Szabó, A. W. van der Vaart and J. H. van Zanten, Bayes procedures for adaptive inference in inverse problems for the white noise model, Probab. Theory Rel., 164 (2016), 771-813. doi: 10.1007/s00440-015-0619-7.
[24]	J. Koponen, T. Huttunen, T. Tarvainen and J. P. Kaipio, Bayesian approximation error approach in full-wave ultrasound tomography, IEEE T. Ultrason. Ferr., 61 (2014), 1627-1637. doi: 10.1109/TUFFC.2014.006319.
[25]	P. Mathé, Bayesian inverse problems with non-commuting operators, Math. Comput., 88 (2019), 2897-2912. doi: 10.1090/mcom/3439.
[26]	G. D. Prato, An Introduction to Infinite-Dimensional Analysis, Scuola Normale Superiore, Pisa, 2001. doi: 10.1007/3-540-29021-4.
[27]	M. A. Shubin and S. I. Andersson, Pseudodifferential Operators and Spectral Theory, Springer, Berlin, 2001.
[28]	A. W. Stuart, Inverse problems: A Bayesian perspective, Acta Numer., 19 (2010), 451-559. doi: 10.1017/S0962492910000061.
[29]	B. Szabó, A. W. van der Vaart and J. H. van Zanten, Frequentist coverage of adaptive nonparametric Bayesian credible sets, Ann. Statist., 43 (2015), 1391-1428. doi: 10.1214/14-AOS1270.
[30]	M. Trabs, Bayesian inverse problems with unknown operators, Inverse Probl., 34 (2018), 085001, 27pp. doi: 10.1088/1361-6420/aac3aa.
[31]	S. J. Vollmer, Posterior consistency for Bayesian inverse problems through stability and regression results, Inverse Probl., 29 (2013), 125011, 32pp. doi: 10.1088/0266-5611/29/12/125011.

Figure 1. Left panels the empirical Bayes posterior mean (red) and the true curve (blue, dashed). Right panels corresponding normalized likelihood for $ \hat{\tilde{\alpha}} $ (regularity index for the artificial diagonal problem). We have $ n = 10^3, 10^5, 10^8, $ and $ 10^{12} $, from top to bottom

Figure Options

Download full-size image

Download as PowerPoint slide

[1]	Kui Lin, Shuai Lu, Peter Mathé. Oracle-type posterior contraction rates in Bayesian inverse problems. Inverse Problems and Imaging, 2015, 9 (3) : 895-915. doi: 10.3934/ipi.2015.9.895
[2]	C. Brändle, F. Quirós, Julio D. Rossi. Non-simultaneous blow-up for a quasilinear parabolic system with reaction at the boundary. Communications on Pure and Applied Analysis, 2005, 4 (3) : 523-536. doi: 10.3934/cpaa.2005.4.523
[3]	Lan Qiao, Sining Zheng. Non-simultaneous blow-up for heat equations with positive-negative sources and coupled boundary flux. Communications on Pure and Applied Analysis, 2007, 6 (4) : 1113-1129. doi: 10.3934/cpaa.2007.6.1113
[4]	Karla Flores-Zarur, William Olvera-Lopez. A Bayesian equilibrium for simultaneous first-price auctions for complementary goods and quasi-linear bids. Journal of Dynamics and Games, 2022 doi: 10.3934/jdg.2022007
[5]	Sari Lasanen. Non-Gaussian statistical inverse problems. Part II: Posterior convergence for approximated unknowns. Inverse Problems and Imaging, 2012, 6 (2) : 267-287. doi: 10.3934/ipi.2012.6.267
[6]	Sari Lasanen. Non-Gaussian statistical inverse problems. Part I: Posterior distributions. Inverse Problems and Imaging, 2012, 6 (2) : 215-266. doi: 10.3934/ipi.2012.6.215
[7]	Jiangfeng Huang, Zhiliang Deng, Liwei Xu. A Bayesian level set method for an inverse medium scattering problem in acoustics. Inverse Problems and Imaging, 2021, 15 (5) : 1077-1097. doi: 10.3934/ipi.2021029
[8]	Mikko Orispää, Markku Lehtinen. Fortran linear inverse problem solver. Inverse Problems and Imaging, 2010, 4 (3) : 485-503. doi: 10.3934/ipi.2010.4.485
[9]	Natalia P. Bondarenko, Vjacheslav A. Yurko. A new approach to the inverse discrete transmission eigenvalue problem. Inverse Problems and Imaging, , () : -. doi: 10.3934/ipi.2021073
[10]	Yunwen Yin, Weishi Yin, Pinchao Meng, Hongyu Liu. The interior inverse scattering problem for a two-layered cavity using the Bayesian method. Inverse Problems and Imaging, , () : -. doi: 10.3934/ipi.2021069
[11]	Bariş Keçeci, Fulya Altıparmak, İmdat Kara. A mathematical formulation and heuristic approach for the heterogeneous fixed fleet vehicle routing problem with simultaneous pickup and delivery. Journal of Industrial and Management Optimization, 2021, 17 (3) : 1069-1100. doi: 10.3934/jimo.2020012
[12]	Masoumeh Dashti, Stephen Harris, Andrew Stuart. Besov priors for Bayesian inverse problems. Inverse Problems and Imaging, 2012, 6 (2) : 183-200. doi: 10.3934/ipi.2012.6.183
[13]	Yi Zhang, Yong Jiang, Liwei Zhang, Jiangzhong Zhang. A perturbation approach for an inverse linear second-order cone programming. Journal of Industrial and Management Optimization, 2013, 9 (1) : 171-189. doi: 10.3934/jimo.2013.9.171
[14]	Shiyun Wang, Yong-Jin Liu, Yong Jiang. A majorized penalty approach to inverse linear second order cone programming problems. Journal of Industrial and Management Optimization, 2014, 10 (3) : 965-976. doi: 10.3934/jimo.2014.10.965
[15]	Yibing Lv, Zhongping Wan. Linear bilevel multiobjective optimization problem: Penalty approach. Journal of Industrial and Management Optimization, 2019, 15 (3) : 1213-1223. doi: 10.3934/jimo.2018092
[16]	William Chad Young, Adrian E. Raftery, Ka Yee Yeung. A posterior probability approach for gene regulatory network inference in genetic perturbation data. Mathematical Biosciences & Engineering, 2016, 13 (6) : 1241-1251. doi: 10.3934/mbe.2016041
[17]	Rakesh Pilkar, Erik M. Bollt, Charles Robinson. Empirical mode decomposition/Hilbert transform analysis of postural responses to small amplitude anterior-posterior sinusoidal translations of varying frequencies. Mathematical Biosciences & Engineering, 2011, 8 (4) : 1085-1097. doi: 10.3934/mbe.2011.8.1085
[18]	Ben Schweizer, Marco Veneroni. The needle problem approach to non-periodic homogenization. Networks and Heterogeneous Media, 2011, 6 (4) : 755-781. doi: 10.3934/nhm.2011.6.755
[19]	Lili Chang, Wei Gong, Guiquan Sun, Ningning Yan. PDE-constrained optimal control approach for the approximation of an inverse Cauchy problem. Inverse Problems and Imaging, 2015, 9 (3) : 791-814. doi: 10.3934/ipi.2015.9.791
[20]	Junfeng Yang. Dynamic power price problem: An inverse variational inequality approach. Journal of Industrial and Management Optimization, 2008, 4 (4) : 673-684. doi: 10.3934/jimo.2008.4.673

2020 Impact Factor: 1.639

Tools

Metrics

Tools

Article outline

Figures and Tables

2020 Impact Factor: 1.639

Tools

Figures and Tables

2020 Impact Factor: 1.639

American Institute of Mathematical Sciences

Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption

References:

References:

Tools

Metrics

Other articles
by authors

Tools

Tools

Tools

Metrics

Other articles
by authors

American Institute of Mathematical Sciences

Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption

References:

References:

Tools

Metrics

Other articlesby authors

Tools

Tools

Tools

Metrics

Other articlesby authors

Other articles
by authors

Other articles
by authors