# American Institute of Mathematical Sciences

October  2019, 13(5): 1113-1137. doi: 10.3934/ipi.2019050

## Kullback-Leibler residual and regularization for inverse problems with noisy data and noisy operator

 CREATIS, CNRS UMR 5220; INSERM U1044; INSA de Lyon; Université de Lyon 1, Université de Lyon, 69621, Villeurbanne Cedex France

Received  March 2019 Revised  May 2019 Published  July 2019

We study the properties of a regularization method for inverse problems with joint Kullback-Leibler data term and regularization when the data and the operator are corrupted by some noise. We show the convergence of the method and we obtain convergence rates for the approximate solution of the inverse problem and for the operator when it is characterized by some kernel, under the assumption that some source conditions are satisfied. Numerical results showing the effect of the noise levels on the reconstructed solution are provided for Spectral Computerized Tomography.

Citation: Bruno Sixou, Cyril Mory. Kullback-Leibler residual and regularization for inverse problems with noisy data and noisy operator. Inverse Problems & Imaging, 2019, 13 (5) : 1113-1137. doi: 10.3934/ipi.2019050
##### References:
 [1] S. W. Anzengruber and R. Ramlau, Discrepancy principle for Tikhonov-type functionals with nonlinear operators, Inverse Problems, 26 (2010), 025001, 17pp. doi: 10.1088/0266-5611/26/2/025001. Google Scholar [2] R. F. Barber, E. Y. Sidky, T. G. Schmidt and X. Pan, An algorithm for constrained one-step of spectral CT data, Physics in Medicine and Biology, 61 (2016), 3784.Google Scholar [3] J. M. Bardsley and J. Goldes, Regularization parameter selection methods for ill-posed Poisson maximum likelihood estimation, Inverse Problems, 25 (2009), 095005, 18pp. doi: 10.1088/0266-5611/25/9/095005. Google Scholar [4] O. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, Wiley Publishers; 1978. Google Scholar [5] M. Bertero, P. Boccacci, C. Talenti, R. Zanella and L. Zanni, A discrepancy principle for Poisson data, Inverse Problems, 25 (2010), 105004, 20pp. doi: 10.1088/0266-5611/26/10/105004. Google Scholar [6] J. R. Bleyer and R. Ramlau, A double regularization approach for inverse problems with noisy data and inexact operator, Inverse Problems, 29 (2013), 025004, 16pp. doi: 10.1088/0266-5611/29/2/025004. Google Scholar [7] I. R. Bleyer and R. Ramlau, An alternating iterative minimisation algorithm for the double-regularised total least square functional, Inverse Problems, 31 (2015), 075004, 21pp. doi: 10.1088/0266-5611/31/7/075004. Google Scholar [8] M. Burger and S. Osher, Convergence rates of convex variational reglarization, Inverse Problems, 10 (2004), 1411-1422. doi: 10.1088/0266-5611/20/5/005. Google Scholar [9] N. Ducros, J. F. P. Abascal, B. Sixou, S. Rit and F. Peyrin, Regularization of nonlinear decomposition of spectral X-ray projection images, Med.Phys., 44 (2017), 174-187. Google Scholar [10] P. P. B. Eggermont, Maximum entropy regularization for Fredholm integral equations of the first kind, SIAM Journal on Mathematical Analysis, 24 (1993), 1557-1576. doi: 10.1137/0524088. Google Scholar [11] H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers: Dordrecht; 1996. Google Scholar [12] G. H. Golub, P. C. Hansen and D. P. O'Leary, Tikhonov regularization and total least squares, SIAM J. Numer. Anal. Appl., 21 (1999), 185-194. doi: 10.1137/S0895479897326432. Google Scholar [13] P. C. Hansen and M. Saxild-Hansen, AIR Tools-A MATLAB package of algebraic iterative reconstruction methods, Journal of Computational and Applied Mathematics, 236 (2012), 2067-2178. doi: 10.1016/j.cam.2011.09.039. Google Scholar [14] B. Hofmann, B. Kaltenbacher, C. Poschl and O. Scherzer, A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators, Inverse Problems, 23 (2007), 987-1010. doi: 10.1088/0266-5611/23/3/009. Google Scholar [15] T. Hohage and F. Werner, Inverse problems with Poisson data: Statistical regularization theory, applications and algorithms, Inverse Problems, 32 (2016), 093001, 56pp. doi: 10.1088/0266-5611/32/9/093001. Google Scholar [16] Y. Long and J. A. Fessler, Multi-material decomposition using statistical image reconstruction for spectral CT, IEEE Transactions on Medical Imaging, 33 (2014), 1614-1626. Google Scholar [17] S. Lu, S. V. Pereverzev and U. Tautenhahn, Regularized total least squares: Computational aspects and error bounds, SIAM J. Numer. Anal. Appl., 31 (2009), 918-941. doi: 10.1137/070709086. Google Scholar [18] S. Lu and J. Flemming, Convergence rate analysis of Tikhonov regularization for nonlinear ill-posed problems with noisy operators, Inverse Problems, 28 (2012), 104003, 19pp. doi: 10.1088/0266-5611/28/10/104003. Google Scholar [19] K. Mechlem, S. Ehen, T. Sellerer, E. Braig, D. Munzel, F. Pfeibber and P. B. Noel, Joint satistical iterative material image reconstruction for spectral computed tomography using a semi-empirical forward model, IEEE Transactions on Medical Imaging, 37 (2018), 68-80. Google Scholar [20] C. Pöschl, Tikhonov Regularization with General Residual Term, PhD Thesis, Univeristat Innsbruck; 2008.Google Scholar [21] E. Resmerita and R. S. Anderssen, Joint additive Kullback-Leibler residual minimization and regularization for linear inverse problem, Mathematical Methods in the Applied Sciences, 30 (2007), 1527-1544. doi: 10.1002/mma.855. Google Scholar [22] E. Resmerita, Regularization of ill-posed inverse problems in Banach spaces:convergence rates, Inverse Problems, 21 (2005), 1301-1314. doi: 10.1088/0266-5611/21/4/007. Google Scholar [23] R. J. Santos and A. R. De Pierro, A new parameters choice method for ill-posed problem with Poisson data and its application to emission tomographic imaging, International Journal of Tomography and Statistics, 11 (2009), 33-52. Google Scholar [24] O. Scherzer, M. Grassmair, H. Grossauer, M. Haltmaier and F. Lenzen, Variational Methods in Imaging, Applied Mathematical Sciences, 167. Springer, New York, 2009. Google Scholar [25] J. P. Schlomka, E. Roessl, R. Dorscheid, S. Dill, G. Martens, T. Istel, C. Bumer, C. Herrmann, R. Steadman, G. Zeitler, A. Livne and R. Proska, Experimental feasiblitiy of multi-energy photon-counting k-edge imaging in pre-clinical computed tomography, Physics in Medicine and Biology, 53 (2008), 4031.Google Scholar [26] U. Tautenhahn, Regularization of linear ill-posed problems with noisy right hand side and noisy operator, SIAM J.Inverse Ill-Posed Problems, 16 (2008), 507-523. doi: 10.1515/JIIP.2008.027. Google Scholar [27] Y. Wang, A. C. Yagola and C. Yang, Optimization and Regularization for Computational Inverse Problems and Applications, Berlin: Springer; 2010.Google Scholar [28] T. Weidinger, T. M. Buzug, T. Flohr, S. Kappler and K. Stierstorfer, Polychromatic iterative satistical material image reconstruction for photon-counting computed tomography, International Journal of Biomedical Imaging, 2 (2016), 1-15. Google Scholar [29] F. Werner and T. Hohage, Convergence rate in expectation ofr Tikhonov-type regularization of inverse problems with Poisson data, Inverse Problems, 28 (2012), 104004, 15pp. doi: 10.1088/0266-5611/28/10/104004. Google Scholar [30] F. Werner, Inverse Problems with Poisson Data: Tikhonov-type Regularization and Iteratively Regularized Newton Methods, PhD thesis, University of Göttingen; 2012.Google Scholar

show all references

##### References:
 [1] S. W. Anzengruber and R. Ramlau, Discrepancy principle for Tikhonov-type functionals with nonlinear operators, Inverse Problems, 26 (2010), 025001, 17pp. doi: 10.1088/0266-5611/26/2/025001. Google Scholar [2] R. F. Barber, E. Y. Sidky, T. G. Schmidt and X. Pan, An algorithm for constrained one-step of spectral CT data, Physics in Medicine and Biology, 61 (2016), 3784.Google Scholar [3] J. M. Bardsley and J. Goldes, Regularization parameter selection methods for ill-posed Poisson maximum likelihood estimation, Inverse Problems, 25 (2009), 095005, 18pp. doi: 10.1088/0266-5611/25/9/095005. Google Scholar [4] O. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, Wiley Publishers; 1978. Google Scholar [5] M. Bertero, P. Boccacci, C. Talenti, R. Zanella and L. Zanni, A discrepancy principle for Poisson data, Inverse Problems, 25 (2010), 105004, 20pp. doi: 10.1088/0266-5611/26/10/105004. Google Scholar [6] J. R. Bleyer and R. Ramlau, A double regularization approach for inverse problems with noisy data and inexact operator, Inverse Problems, 29 (2013), 025004, 16pp. doi: 10.1088/0266-5611/29/2/025004. Google Scholar [7] I. R. Bleyer and R. Ramlau, An alternating iterative minimisation algorithm for the double-regularised total least square functional, Inverse Problems, 31 (2015), 075004, 21pp. doi: 10.1088/0266-5611/31/7/075004. Google Scholar [8] M. Burger and S. Osher, Convergence rates of convex variational reglarization, Inverse Problems, 10 (2004), 1411-1422. doi: 10.1088/0266-5611/20/5/005. Google Scholar [9] N. Ducros, J. F. P. Abascal, B. Sixou, S. Rit and F. Peyrin, Regularization of nonlinear decomposition of spectral X-ray projection images, Med.Phys., 44 (2017), 174-187. Google Scholar [10] P. P. B. Eggermont, Maximum entropy regularization for Fredholm integral equations of the first kind, SIAM Journal on Mathematical Analysis, 24 (1993), 1557-1576. doi: 10.1137/0524088. Google Scholar [11] H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers: Dordrecht; 1996. Google Scholar [12] G. H. Golub, P. C. Hansen and D. P. O'Leary, Tikhonov regularization and total least squares, SIAM J. Numer. Anal. Appl., 21 (1999), 185-194. doi: 10.1137/S0895479897326432. Google Scholar [13] P. C. Hansen and M. Saxild-Hansen, AIR Tools-A MATLAB package of algebraic iterative reconstruction methods, Journal of Computational and Applied Mathematics, 236 (2012), 2067-2178. doi: 10.1016/j.cam.2011.09.039. Google Scholar [14] B. Hofmann, B. Kaltenbacher, C. Poschl and O. Scherzer, A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators, Inverse Problems, 23 (2007), 987-1010. doi: 10.1088/0266-5611/23/3/009. Google Scholar [15] T. Hohage and F. Werner, Inverse problems with Poisson data: Statistical regularization theory, applications and algorithms, Inverse Problems, 32 (2016), 093001, 56pp. doi: 10.1088/0266-5611/32/9/093001. Google Scholar [16] Y. Long and J. A. Fessler, Multi-material decomposition using statistical image reconstruction for spectral CT, IEEE Transactions on Medical Imaging, 33 (2014), 1614-1626. Google Scholar [17] S. Lu, S. V. Pereverzev and U. Tautenhahn, Regularized total least squares: Computational aspects and error bounds, SIAM J. Numer. Anal. Appl., 31 (2009), 918-941. doi: 10.1137/070709086. Google Scholar [18] S. Lu and J. Flemming, Convergence rate analysis of Tikhonov regularization for nonlinear ill-posed problems with noisy operators, Inverse Problems, 28 (2012), 104003, 19pp. doi: 10.1088/0266-5611/28/10/104003. Google Scholar [19] K. Mechlem, S. Ehen, T. Sellerer, E. Braig, D. Munzel, F. Pfeibber and P. B. Noel, Joint satistical iterative material image reconstruction for spectral computed tomography using a semi-empirical forward model, IEEE Transactions on Medical Imaging, 37 (2018), 68-80. Google Scholar [20] C. Pöschl, Tikhonov Regularization with General Residual Term, PhD Thesis, Univeristat Innsbruck; 2008.Google Scholar [21] E. Resmerita and R. S. Anderssen, Joint additive Kullback-Leibler residual minimization and regularization for linear inverse problem, Mathematical Methods in the Applied Sciences, 30 (2007), 1527-1544. doi: 10.1002/mma.855. Google Scholar [22] E. Resmerita, Regularization of ill-posed inverse problems in Banach spaces:convergence rates, Inverse Problems, 21 (2005), 1301-1314. doi: 10.1088/0266-5611/21/4/007. Google Scholar [23] R. J. Santos and A. R. De Pierro, A new parameters choice method for ill-posed problem with Poisson data and its application to emission tomographic imaging, International Journal of Tomography and Statistics, 11 (2009), 33-52. Google Scholar [24] O. Scherzer, M. Grassmair, H. Grossauer, M. Haltmaier and F. Lenzen, Variational Methods in Imaging, Applied Mathematical Sciences, 167. Springer, New York, 2009. Google Scholar [25] J. P. Schlomka, E. Roessl, R. Dorscheid, S. Dill, G. Martens, T. Istel, C. Bumer, C. Herrmann, R. Steadman, G. Zeitler, A. Livne and R. Proska, Experimental feasiblitiy of multi-energy photon-counting k-edge imaging in pre-clinical computed tomography, Physics in Medicine and Biology, 53 (2008), 4031.Google Scholar [26] U. Tautenhahn, Regularization of linear ill-posed problems with noisy right hand side and noisy operator, SIAM J.Inverse Ill-Posed Problems, 16 (2008), 507-523. doi: 10.1515/JIIP.2008.027. Google Scholar [27] Y. Wang, A. C. Yagola and C. Yang, Optimization and Regularization for Computational Inverse Problems and Applications, Berlin: Springer; 2010.Google Scholar [28] T. Weidinger, T. M. Buzug, T. Flohr, S. Kappler and K. Stierstorfer, Polychromatic iterative satistical material image reconstruction for photon-counting computed tomography, International Journal of Biomedical Imaging, 2 (2016), 1-15. Google Scholar [29] F. Werner and T. Hohage, Convergence rate in expectation ofr Tikhonov-type regularization of inverse problems with Poisson data, Inverse Problems, 28 (2012), 104004, 15pp. doi: 10.1088/0266-5611/28/10/104004. Google Scholar [30] F. Werner, Inverse Problems with Poisson Data: Tikhonov-type Regularization and Iteratively Regularized Newton Methods, PhD thesis, University of Göttingen; 2012.Google Scholar
Ground truth maps (a) iodine (b) gadolinium (c) water
Detector response functions $d_i(E)$ for the five energy bins
Evolution of the data term for different noise levels. Bold line ($\delta^{'} = 0.1 , \delta_F' = 0$), dashed line ($\delta' = 0.1 , \delta_F' = 0.1$), thin line ($\delta' = 0.1 , \delta_F' = 0.2$)
Evolution of the iodine relative reconstruction error. Bold line ($\delta' = 0.1 , \delta_F' = 0$), dashed line ($\delta' = 0.1 , \delta_F' = 0.1$), thin line ($\delta' = 0.1 , \delta_F' = 0.2$)
Evolution of the gadolinium relative reconstruction error. Bold line ($\delta^{'} = 0.1 , \delta_F' = 0$), dashed line ($\delta^{'} = 0.1 , \delta_F' = 0.1$), thin line ($\delta^{'} = 0.1 , \delta_F' = 0.2$)
Evolution of the water relative reconstruction error. Bold line ($\delta' = 0.1 , \delta_F' = 0$), dashed line ($\delta = 0.1 , \delta_F' = 0.1$), thin line ($\delta' = 0.1 , \delta_F' = 0.2$)
Reconstruction maps for the noise levels (a) iodine (b) gadolinium (c) water ($\delta^{'} = 0.1, \delta_F = 0.1$)
Reconstruction maps for the noise levels (a) iodine (b) gadolinium (c) water ($\delta^{'} = 0.1 , \delta_F = 0.2$)
Evolution of the water reconstruction error. Bold line ($\delta^{'} = 0.1 , \delta_F = 0$), thin line ($\delta^{'} = 0.1 , \delta_F' = 0.1$), dashed line: solution obtained with alternate minimization
Evolution of the iodine reconstruction error. Bold line ($\delta^{'} = 0.1 , \delta_F = 0$), thin line ($\delta^{'} = 0.1 , \delta_F' = 0.1$), dashed line: solution obtained with alternate minimization
Evolution of the gadolinium relative reconstruction error. Bold line ($\delta^{'} = 0.1 , \delta_F = 0$), thin line ($\delta^{'} = 0.05 , \delta_F' = 0.1$), dashed line: solution obtained with alternate minimization
Reconstruction maps for (a) iodine (b) gadolinium (c) water obtained with the iterative algorithm starting form ($\delta' = 0.1 , \delta_F = 0.1$
 [1] Bruno Sixou, Tom Hohweiller, Nicolas Ducros. Morozov principle for Kullback-Leibler residual term and Poisson noise. Inverse Problems & Imaging, 2018, 12 (3) : 607-634. doi: 10.3934/ipi.2018026 [2] Raphaël Danchin, Piotr B. Mucha. Divergence. Discrete & Continuous Dynamical Systems - S, 2013, 6 (5) : 1163-1172. doi: 10.3934/dcdss.2013.6.1163 [3] Gary Lieberman. Nonlocal problems for quasilinear parabolic equations in divergence form. Conference Publications, 2003, 2003 (Special) : 563-570. doi: 10.3934/proc.2003.2003.563 [4] Maria Rosaria Lancia, Valerio Regis Durante, Paola Vernole. Asymptotics for Venttsel' problems for operators in non divergence form in irregular domains. Discrete & Continuous Dynamical Systems - S, 2016, 9 (5) : 1493-1520. doi: 10.3934/dcdss.2016060 [5] Colin Guillarmou, Antônio Sá Barreto. Inverse problems for Einstein manifolds. Inverse Problems & Imaging, 2009, 3 (1) : 1-15. doi: 10.3934/ipi.2009.3.1 [6] Sergei Avdonin, Pavel Kurasov. Inverse problems for quantum trees. Inverse Problems & Imaging, 2008, 2 (1) : 1-21. doi: 10.3934/ipi.2008.2.1 [7] Maciej Zworski. A remark on inverse problems for resonances. Inverse Problems & Imaging, 2007, 1 (1) : 225-227. doi: 10.3934/ipi.2007.1.225 [8] Guanghui Hu, Peijun Li, Xiaodong Liu, Yue Zhao. Inverse source problems in electrodynamics. Inverse Problems & Imaging, 2018, 12 (6) : 1411-1428. doi: 10.3934/ipi.2018059 [9] Michael Herty, Giuseppe Visconti. Kinetic methods for inverse problems. Kinetic & Related Models, 2019, 12 (5) : 1109-1130. doi: 10.3934/krm.2019042 [10] Janne M.J. Huttunen, J. P. Kaipio. Approximation errors in nonstationary inverse problems. Inverse Problems & Imaging, 2007, 1 (1) : 77-93. doi: 10.3934/ipi.2007.1.77 [11] Masoumeh Dashti, Stephen Harris, Andrew Stuart. Besov priors for Bayesian inverse problems. Inverse Problems & Imaging, 2012, 6 (2) : 183-200. doi: 10.3934/ipi.2012.6.183 [12] Xiaosheng Li, Gunther Uhlmann. Inverse problems with partial data in a slab. Inverse Problems & Imaging, 2010, 4 (3) : 449-462. doi: 10.3934/ipi.2010.4.449 [13] Sergei A. Avdonin, Sergei A. Ivanov, Jun-Min Wang. Inverse problems for the heat equation with memory. Inverse Problems & Imaging, 2019, 13 (1) : 31-38. doi: 10.3934/ipi.2019002 [14] Johannes Elschner, Guanghui Hu. Uniqueness in inverse transmission scattering problems for multilayered obstacles. Inverse Problems & Imaging, 2011, 5 (4) : 793-813. doi: 10.3934/ipi.2011.5.793 [15] François Monard, Guillaume Bal. Inverse diffusion problems with redundant internal information. Inverse Problems & Imaging, 2012, 6 (2) : 289-313. doi: 10.3934/ipi.2012.6.289 [16] Gabriel Peyré, Sébastien Bougleux, Laurent Cohen. Non-local regularization of inverse problems. Inverse Problems & Imaging, 2011, 5 (2) : 511-530. doi: 10.3934/ipi.2011.5.511 [17] Simopekka Vänskä. Stationary waves method for inverse scattering problems. Inverse Problems & Imaging, 2008, 2 (4) : 577-586. doi: 10.3934/ipi.2008.2.577 [18] Victor Isakov, Shuai Lu. Inverse source problems without (pseudo) convexity assumptions. Inverse Problems & Imaging, 2018, 12 (4) : 955-970. doi: 10.3934/ipi.2018040 [19] Sergei Avdonin, Fritz Gesztesy, Konstantin A. Makarov. Spectral estimation and inverse initial boundary value problems. Inverse Problems & Imaging, 2010, 4 (1) : 1-9. doi: 10.3934/ipi.2010.4.1 [20] Johnathan M. Bardsley. Gaussian Markov random field priors for inverse problems. Inverse Problems & Imaging, 2013, 7 (2) : 397-416. doi: 10.3934/ipi.2013.7.397

2018 Impact Factor: 1.469

## Tools

Article outline

Figures and Tables

[Back to Top]