October  2019, 13(5): 1113-1137. doi: 10.3934/ipi.2019050

Kullback-Leibler residual and regularization for inverse problems with noisy data and noisy operator

CREATIS, CNRS UMR 5220; INSERM U1044; INSA de Lyon; Université de Lyon 1, Université de Lyon, 69621, Villeurbanne Cedex France

Received  March 2019 Revised  May 2019 Published  July 2019

We study the properties of a regularization method for inverse problems with joint Kullback-Leibler data term and regularization when the data and the operator are corrupted by some noise. We show the convergence of the method and we obtain convergence rates for the approximate solution of the inverse problem and for the operator when it is characterized by some kernel, under the assumption that some source conditions are satisfied. Numerical results showing the effect of the noise levels on the reconstructed solution are provided for Spectral Computerized Tomography.

Citation: Bruno Sixou, Cyril Mory. Kullback-Leibler residual and regularization for inverse problems with noisy data and noisy operator. Inverse Problems and Imaging, 2019, 13 (5) : 1113-1137. doi: 10.3934/ipi.2019050
References:
[1]

S. W. Anzengruber and R. Ramlau, Discrepancy principle for Tikhonov-type functionals with nonlinear operators, Inverse Problems, 26 (2010), 025001, 17pp. doi: 10.1088/0266-5611/26/2/025001.

[2]

R. F. Barber, E. Y. Sidky, T. G. Schmidt and X. Pan, An algorithm for constrained one-step of spectral CT data, Physics in Medicine and Biology, 61 (2016), 3784.

[3]

J. M. Bardsley and J. Goldes, Regularization parameter selection methods for ill-posed Poisson maximum likelihood estimation, Inverse Problems, 25 (2009), 095005, 18pp. doi: 10.1088/0266-5611/25/9/095005.

[4]

O. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, Wiley Publishers; 1978.

[5]

M. Bertero, P. Boccacci, C. Talenti, R. Zanella and L. Zanni, A discrepancy principle for Poisson data, Inverse Problems, 25 (2010), 105004, 20pp. doi: 10.1088/0266-5611/26/10/105004.

[6]

J. R. Bleyer and R. Ramlau, A double regularization approach for inverse problems with noisy data and inexact operator, Inverse Problems, 29 (2013), 025004, 16pp. doi: 10.1088/0266-5611/29/2/025004.

[7]

I. R. Bleyer and R. Ramlau, An alternating iterative minimisation algorithm for the double-regularised total least square functional, Inverse Problems, 31 (2015), 075004, 21pp. doi: 10.1088/0266-5611/31/7/075004.

[8]

M. Burger and S. Osher, Convergence rates of convex variational reglarization, Inverse Problems, 10 (2004), 1411-1422.  doi: 10.1088/0266-5611/20/5/005.

[9]

N. DucrosJ. F. P. AbascalB. SixouS. Rit and F. Peyrin, Regularization of nonlinear decomposition of spectral X-ray projection images, Med.Phys., 44 (2017), 174-187. 

[10]

P. P. B. Eggermont, Maximum entropy regularization for Fredholm integral equations of the first kind, SIAM Journal on Mathematical Analysis, 24 (1993), 1557-1576.  doi: 10.1137/0524088.

[11]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers: Dordrecht; 1996.

[12]

G. H. GolubP. C. Hansen and D. P. O'Leary, Tikhonov regularization and total least squares, SIAM J. Numer. Anal. Appl., 21 (1999), 185-194.  doi: 10.1137/S0895479897326432.

[13]

P. C. Hansen and M. Saxild-Hansen, AIR Tools-A MATLAB package of algebraic iterative reconstruction methods, Journal of Computational and Applied Mathematics, 236 (2012), 2067-2178.  doi: 10.1016/j.cam.2011.09.039.

[14]

B. HofmannB. KaltenbacherC. Poschl and O. Scherzer, A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators, Inverse Problems, 23 (2007), 987-1010.  doi: 10.1088/0266-5611/23/3/009.

[15]

T. Hohage and F. Werner, Inverse problems with Poisson data: Statistical regularization theory, applications and algorithms, Inverse Problems, 32 (2016), 093001, 56pp. doi: 10.1088/0266-5611/32/9/093001.

[16]

Y. Long and J. A. Fessler, Multi-material decomposition using statistical image reconstruction for spectral CT, IEEE Transactions on Medical Imaging, 33 (2014), 1614-1626. 

[17]

S. LuS. V. Pereverzev and U. Tautenhahn, Regularized total least squares: Computational aspects and error bounds, SIAM J. Numer. Anal. Appl., 31 (2009), 918-941.  doi: 10.1137/070709086.

[18]

S. Lu and J. Flemming, Convergence rate analysis of Tikhonov regularization for nonlinear ill-posed problems with noisy operators, Inverse Problems, 28 (2012), 104003, 19pp. doi: 10.1088/0266-5611/28/10/104003.

[19]

K. MechlemS. EhenT. SellererE. BraigD. MunzelF. Pfeibber and P. B. Noel, Joint satistical iterative material image reconstruction for spectral computed tomography using a semi-empirical forward model, IEEE Transactions on Medical Imaging, 37 (2018), 68-80. 

[20]

C. Pöschl, Tikhonov Regularization with General Residual Term, PhD Thesis, Univeristat Innsbruck; 2008.

[21]

E. Resmerita and R. S. Anderssen, Joint additive Kullback-Leibler residual minimization and regularization for linear inverse problem, Mathematical Methods in the Applied Sciences, 30 (2007), 1527-1544.  doi: 10.1002/mma.855.

[22]

E. Resmerita, Regularization of ill-posed inverse problems in Banach spaces:convergence rates, Inverse Problems, 21 (2005), 1301-1314.  doi: 10.1088/0266-5611/21/4/007.

[23]

R. J. Santos and A. R. De Pierro, A new parameters choice method for ill-posed problem with Poisson data and its application to emission tomographic imaging, International Journal of Tomography and Statistics, 11 (2009), 33-52. 

[24]

O. Scherzer, M. Grassmair, H. Grossauer, M. Haltmaier and F. Lenzen, Variational Methods in Imaging, Applied Mathematical Sciences, 167. Springer, New York, 2009.

[25]

J. P. Schlomka, E. Roessl, R. Dorscheid, S. Dill, G. Martens, T. Istel, C. Bumer, C. Herrmann, R. Steadman, G. Zeitler, A. Livne and R. Proska, Experimental feasiblitiy of multi-energy photon-counting k-edge imaging in pre-clinical computed tomography, Physics in Medicine and Biology, 53 (2008), 4031.

[26]

U. Tautenhahn, Regularization of linear ill-posed problems with noisy right hand side and noisy operator, SIAM J.Inverse Ill-Posed Problems, 16 (2008), 507-523.  doi: 10.1515/JIIP.2008.027.

[27]

Y. Wang, A. C. Yagola and C. Yang, Optimization and Regularization for Computational Inverse Problems and Applications, Berlin: Springer; 2010.

[28]

T. WeidingerT. M. BuzugT. FlohrS. Kappler and K. Stierstorfer, Polychromatic iterative satistical material image reconstruction for photon-counting computed tomography, International Journal of Biomedical Imaging, 2 (2016), 1-15. 

[29]

F. Werner and T. Hohage, Convergence rate in expectation ofr Tikhonov-type regularization of inverse problems with Poisson data, Inverse Problems, 28 (2012), 104004, 15pp. doi: 10.1088/0266-5611/28/10/104004.

[30]

F. Werner, Inverse Problems with Poisson Data: Tikhonov-type Regularization and Iteratively Regularized Newton Methods, PhD thesis, University of Göttingen; 2012.

show all references

References:
[1]

S. W. Anzengruber and R. Ramlau, Discrepancy principle for Tikhonov-type functionals with nonlinear operators, Inverse Problems, 26 (2010), 025001, 17pp. doi: 10.1088/0266-5611/26/2/025001.

[2]

R. F. Barber, E. Y. Sidky, T. G. Schmidt and X. Pan, An algorithm for constrained one-step of spectral CT data, Physics in Medicine and Biology, 61 (2016), 3784.

[3]

J. M. Bardsley and J. Goldes, Regularization parameter selection methods for ill-posed Poisson maximum likelihood estimation, Inverse Problems, 25 (2009), 095005, 18pp. doi: 10.1088/0266-5611/25/9/095005.

[4]

O. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, Wiley Publishers; 1978.

[5]

M. Bertero, P. Boccacci, C. Talenti, R. Zanella and L. Zanni, A discrepancy principle for Poisson data, Inverse Problems, 25 (2010), 105004, 20pp. doi: 10.1088/0266-5611/26/10/105004.

[6]

J. R. Bleyer and R. Ramlau, A double regularization approach for inverse problems with noisy data and inexact operator, Inverse Problems, 29 (2013), 025004, 16pp. doi: 10.1088/0266-5611/29/2/025004.

[7]

I. R. Bleyer and R. Ramlau, An alternating iterative minimisation algorithm for the double-regularised total least square functional, Inverse Problems, 31 (2015), 075004, 21pp. doi: 10.1088/0266-5611/31/7/075004.

[8]

M. Burger and S. Osher, Convergence rates of convex variational reglarization, Inverse Problems, 10 (2004), 1411-1422.  doi: 10.1088/0266-5611/20/5/005.

[9]

N. DucrosJ. F. P. AbascalB. SixouS. Rit and F. Peyrin, Regularization of nonlinear decomposition of spectral X-ray projection images, Med.Phys., 44 (2017), 174-187. 

[10]

P. P. B. Eggermont, Maximum entropy regularization for Fredholm integral equations of the first kind, SIAM Journal on Mathematical Analysis, 24 (1993), 1557-1576.  doi: 10.1137/0524088.

[11]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers: Dordrecht; 1996.

[12]

G. H. GolubP. C. Hansen and D. P. O'Leary, Tikhonov regularization and total least squares, SIAM J. Numer. Anal. Appl., 21 (1999), 185-194.  doi: 10.1137/S0895479897326432.

[13]

P. C. Hansen and M. Saxild-Hansen, AIR Tools-A MATLAB package of algebraic iterative reconstruction methods, Journal of Computational and Applied Mathematics, 236 (2012), 2067-2178.  doi: 10.1016/j.cam.2011.09.039.

[14]

B. HofmannB. KaltenbacherC. Poschl and O. Scherzer, A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators, Inverse Problems, 23 (2007), 987-1010.  doi: 10.1088/0266-5611/23/3/009.

[15]

T. Hohage and F. Werner, Inverse problems with Poisson data: Statistical regularization theory, applications and algorithms, Inverse Problems, 32 (2016), 093001, 56pp. doi: 10.1088/0266-5611/32/9/093001.

[16]

Y. Long and J. A. Fessler, Multi-material decomposition using statistical image reconstruction for spectral CT, IEEE Transactions on Medical Imaging, 33 (2014), 1614-1626. 

[17]

S. LuS. V. Pereverzev and U. Tautenhahn, Regularized total least squares: Computational aspects and error bounds, SIAM J. Numer. Anal. Appl., 31 (2009), 918-941.  doi: 10.1137/070709086.

[18]

S. Lu and J. Flemming, Convergence rate analysis of Tikhonov regularization for nonlinear ill-posed problems with noisy operators, Inverse Problems, 28 (2012), 104003, 19pp. doi: 10.1088/0266-5611/28/10/104003.

[19]

K. MechlemS. EhenT. SellererE. BraigD. MunzelF. Pfeibber and P. B. Noel, Joint satistical iterative material image reconstruction for spectral computed tomography using a semi-empirical forward model, IEEE Transactions on Medical Imaging, 37 (2018), 68-80. 

[20]

C. Pöschl, Tikhonov Regularization with General Residual Term, PhD Thesis, Univeristat Innsbruck; 2008.

[21]

E. Resmerita and R. S. Anderssen, Joint additive Kullback-Leibler residual minimization and regularization for linear inverse problem, Mathematical Methods in the Applied Sciences, 30 (2007), 1527-1544.  doi: 10.1002/mma.855.

[22]

E. Resmerita, Regularization of ill-posed inverse problems in Banach spaces:convergence rates, Inverse Problems, 21 (2005), 1301-1314.  doi: 10.1088/0266-5611/21/4/007.

[23]

R. J. Santos and A. R. De Pierro, A new parameters choice method for ill-posed problem with Poisson data and its application to emission tomographic imaging, International Journal of Tomography and Statistics, 11 (2009), 33-52. 

[24]

O. Scherzer, M. Grassmair, H. Grossauer, M. Haltmaier and F. Lenzen, Variational Methods in Imaging, Applied Mathematical Sciences, 167. Springer, New York, 2009.

[25]

J. P. Schlomka, E. Roessl, R. Dorscheid, S. Dill, G. Martens, T. Istel, C. Bumer, C. Herrmann, R. Steadman, G. Zeitler, A. Livne and R. Proska, Experimental feasiblitiy of multi-energy photon-counting k-edge imaging in pre-clinical computed tomography, Physics in Medicine and Biology, 53 (2008), 4031.

[26]

U. Tautenhahn, Regularization of linear ill-posed problems with noisy right hand side and noisy operator, SIAM J.Inverse Ill-Posed Problems, 16 (2008), 507-523.  doi: 10.1515/JIIP.2008.027.

[27]

Y. Wang, A. C. Yagola and C. Yang, Optimization and Regularization for Computational Inverse Problems and Applications, Berlin: Springer; 2010.

[28]

T. WeidingerT. M. BuzugT. FlohrS. Kappler and K. Stierstorfer, Polychromatic iterative satistical material image reconstruction for photon-counting computed tomography, International Journal of Biomedical Imaging, 2 (2016), 1-15. 

[29]

F. Werner and T. Hohage, Convergence rate in expectation ofr Tikhonov-type regularization of inverse problems with Poisson data, Inverse Problems, 28 (2012), 104004, 15pp. doi: 10.1088/0266-5611/28/10/104004.

[30]

F. Werner, Inverse Problems with Poisson Data: Tikhonov-type Regularization and Iteratively Regularized Newton Methods, PhD thesis, University of Göttingen; 2012.

Figure 1.  Ground truth maps (a) iodine (b) gadolinium (c) water
Figure 2.  Detector response functions $ d_i(E) $ for the five energy bins
Figure 3.  Evolution of the data term for different noise levels. Bold line ($ \delta^{'} = 0.1 , \delta_F' = 0 $), dashed line ($ \delta' = 0.1 , \delta_F' = 0.1 $), thin line ($ \delta' = 0.1 , \delta_F' = 0.2 $)
Figure 4.  Evolution of the iodine relative reconstruction error. Bold line ($ \delta' = 0.1 , \delta_F' = 0 $), dashed line ($ \delta' = 0.1 , \delta_F' = 0.1 $), thin line ($ \delta' = 0.1 , \delta_F' = 0.2 $)
Figure 5.  Evolution of the gadolinium relative reconstruction error. Bold line ($ \delta^{'} = 0.1 , \delta_F' = 0 $), dashed line ($ \delta^{'} = 0.1 , \delta_F' = 0.1 $), thin line ($ \delta^{'} = 0.1 , \delta_F' = 0.2 $)
Figure 6.  Evolution of the water relative reconstruction error. Bold line ($ \delta' = 0.1 , \delta_F' = 0 $), dashed line ($ \delta = 0.1 , \delta_F' = 0.1 $), thin line ($ \delta' = 0.1 , \delta_F' = 0.2 $)
Figure 7.  Reconstruction maps for the noise levels (a) iodine (b) gadolinium (c) water ($ \delta^{'} = 0.1, \delta_F = 0.1 $)
Figure 8.  Reconstruction maps for the noise levels (a) iodine (b) gadolinium (c) water ($ \delta^{'} = 0.1 , \delta_F = 0.2 $)
Figure 9.  Evolution of the water reconstruction error. Bold line ($ \delta^{'} = 0.1 , \delta_F = 0 $), thin line ($ \delta^{'} = 0.1 , \delta_F' = 0.1 $), dashed line: solution obtained with alternate minimization
Figure 10.  Evolution of the iodine reconstruction error. Bold line ($ \delta^{'} = 0.1 , \delta_F = 0 $), thin line ($ \delta^{'} = 0.1 , \delta_F' = 0.1 $), dashed line: solution obtained with alternate minimization
Figure 11.  Evolution of the gadolinium relative reconstruction error. Bold line ($ \delta^{'} = 0.1 , \delta_F = 0 $), thin line ($ \delta^{'} = 0.05 , \delta_F' = 0.1 $), dashed line: solution obtained with alternate minimization
Figure 12.  Reconstruction maps for (a) iodine (b) gadolinium (c) water obtained with the iterative algorithm starting form ($ \delta' = 0.1 , \delta_F = 0.1 $
[1]

Artur O. Lopes, Jairo K. Mengue. On information gain, Kullback-Leibler divergence, entropy production and the involution kernel. Discrete and Continuous Dynamical Systems, 2022, 42 (7) : 3593-3627. doi: 10.3934/dcds.2022026

[2]

Bruno Sixou, Tom Hohweiller, Nicolas Ducros. Morozov principle for Kullback-Leibler residual term and Poisson noise. Inverse Problems and Imaging, 2018, 12 (3) : 607-634. doi: 10.3934/ipi.2018026

[3]

Raphaël Danchin, Piotr B. Mucha. Divergence. Discrete and Continuous Dynamical Systems - S, 2013, 6 (5) : 1163-1172. doi: 10.3934/dcdss.2013.6.1163

[4]

Gary Lieberman. Nonlocal problems for quasilinear parabolic equations in divergence form. Conference Publications, 2003, 2003 (Special) : 563-570. doi: 10.3934/proc.2003.2003.563

[5]

Maria Rosaria Lancia, Valerio Regis Durante, Paola Vernole. Asymptotics for Venttsel' problems for operators in non divergence form in irregular domains. Discrete and Continuous Dynamical Systems - S, 2016, 9 (5) : 1493-1520. doi: 10.3934/dcdss.2016060

[6]

Michael Herty, Giuseppe Visconti. Kinetic methods for inverse problems. Kinetic and Related Models, 2019, 12 (5) : 1109-1130. doi: 10.3934/krm.2019042

[7]

Guanghui Hu, Peijun Li, Xiaodong Liu, Yue Zhao. Inverse source problems in electrodynamics. Inverse Problems and Imaging, 2018, 12 (6) : 1411-1428. doi: 10.3934/ipi.2018059

[8]

Colin Guillarmou, Antônio Sá Barreto. Inverse problems for Einstein manifolds. Inverse Problems and Imaging, 2009, 3 (1) : 1-15. doi: 10.3934/ipi.2009.3.1

[9]

Sergei Avdonin, Pavel Kurasov. Inverse problems for quantum trees. Inverse Problems and Imaging, 2008, 2 (1) : 1-21. doi: 10.3934/ipi.2008.2.1

[10]

Maciej Zworski. A remark on inverse problems for resonances. Inverse Problems and Imaging, 2007, 1 (1) : 225-227. doi: 10.3934/ipi.2007.1.225

[11]

Janne M.J. Huttunen, J. P. Kaipio. Approximation errors in nonstationary inverse problems. Inverse Problems and Imaging, 2007, 1 (1) : 77-93. doi: 10.3934/ipi.2007.1.77

[12]

Gunther Uhlmann, Jian Zhai. Inverse problems for nonlinear hyperbolic equations. Discrete and Continuous Dynamical Systems, 2021, 41 (1) : 455-469. doi: 10.3934/dcds.2020380

[13]

Masoumeh Dashti, Stephen Harris, Andrew Stuart. Besov priors for Bayesian inverse problems. Inverse Problems and Imaging, 2012, 6 (2) : 183-200. doi: 10.3934/ipi.2012.6.183

[14]

Xiaosheng Li, Gunther Uhlmann. Inverse problems with partial data in a slab. Inverse Problems and Imaging, 2010, 4 (3) : 449-462. doi: 10.3934/ipi.2010.4.449

[15]

Sergei A. Avdonin, Sergei A. Ivanov, Jun-Min Wang. Inverse problems for the heat equation with memory. Inverse Problems and Imaging, 2019, 13 (1) : 31-38. doi: 10.3934/ipi.2019002

[16]

Tony Liimatainen, Lauri Oksanen. Counterexamples to inverse problems for the wave equation. Inverse Problems and Imaging, 2022, 16 (2) : 467-479. doi: 10.3934/ipi.2021058

[17]

François Monard, Guillaume Bal. Inverse diffusion problems with redundant internal information. Inverse Problems and Imaging, 2012, 6 (2) : 289-313. doi: 10.3934/ipi.2012.6.289

[18]

Johannes Elschner, Guanghui Hu. Uniqueness in inverse transmission scattering problems for multilayered obstacles. Inverse Problems and Imaging, 2011, 5 (4) : 793-813. doi: 10.3934/ipi.2011.5.793

[19]

Gabriel Peyré, Sébastien Bougleux, Laurent Cohen. Non-local regularization of inverse problems. Inverse Problems and Imaging, 2011, 5 (2) : 511-530. doi: 10.3934/ipi.2011.5.511

[20]

Simopekka Vänskä. Stationary waves method for inverse scattering problems. Inverse Problems and Imaging, 2008, 2 (4) : 577-586. doi: 10.3934/ipi.2008.2.577

2021 Impact Factor: 1.483

Metrics

  • PDF downloads (322)
  • HTML views (250)
  • Cited by (0)

Other articles
by authors

[Back to Top]