Deep image prior with sparsity constraint for limited-angle computed tomography reconstruction

Leonardo A. Ferreira; Roberto G. Beraldo; Ricardo Suyama; André K. Takahata; John A. Sims

doi:10.3934/ammc.2023009

Article Contents

2023, Volume 1, Issue 2: 105-125. Doi: 10.3934/ammc.2023009

This issue Previous Article Model-based deep learning approaches to the Helsinki Tomography Challenge 2022 Next Article Limited-angle tomography reconstruction via deep end-to-end learning on synthetic data

Deep image prior with sparsity constraint for limited-angle computed tomography reconstruction

1.
Information Engineering, Center for Engineering, Modeling and Applied Social Sciences, Federal University of ABC - 09210-580, Av. dos Estados, 5001 - Bairro Bangú, Santo André - SP, Brazil
2.
AI R & D Lab, Samsung R&D Institute Brazil - 13097-104, Av. Cambacicas, 1200 - Parque Rural Fazenda Santa Cândida, Campinas - SP, Brazil
3.
Biomedical Engineering, Center for Engineering, Modeling and Applied Social Sciences, Federal University of ABC - 09606-045, Alameda da Universidade, s/nº - Bairro Anchieta, São Bernardo do Campo - SP, Brazil

^*Corresponding author: Leonardo A. Ferreira
^*Corresponding author: Leonardo A. Ferreira

Received: April 2023

Revised: November 2023

Early access: November 2023

Published: December 2023

Abstract / Introduction Full Text(HTML) Figure(10) / Table(4) Related Papers Cited by

Abstract

Computed tomography (CT) from full-angle data is of great importance to medical imaging and other areas. However, restrictions during data acquisition, such as measuring the target in a limited angular range, can lead to poorer quality results due to the increased difficulty of the image reconstruction problem. For challenging situations like this, traditional techniques may not be able to produce CT images with enough quality. In this study, we propose software for reconstruction of limited-angle CT images using a combination of deep image prior (DIP) and matching pursuit (MMP). Evaluation of our proposal was performed on the dataset provided by the Helsinki Tomographic Challenge 2022 (HTC 2022). Our results indicate that, by using MMP as a regularizer for the problem, the quality of DIP image reconstructions can be improved. Furthermore, we obtained images with a minimum Matthews correlation coefficient of 0.936 in five of seven difficulty levels of the HTC 2022, showing that it is a promising technique for this application.

Keywords:

Mathematics Subject Classification: 68T07, 68U10.

Citation:

FullText(HTML)

Figure 1. Overview of the proposed technique

Download: Full-size image PowerPoint slide

Figure 2. Example of output images and sinograms throughout the iterations of the outer loop

Download: Full-size image PowerPoint slide

Figure 4. Example of reconstructions with the proposed method (DIP+MMP) in the lower levels of the challenge

Download: Full-size image PowerPoint slide

Figure 6. Example of reconstructions and their corresponding masks for an image from level 5 of the test dataset

Download: Full-size image PowerPoint slide

Figure 8. Example of reconstructions for an image (from level 4 of the test dataset) where the MCC metric was better for the method without MMP

Download: Full-size image PowerPoint slide

Figure 10. Example of reconstructions of an image from level 1, including the reconstruction of the limited-angle data using FBP

Download: Full-size image PowerPoint slide

Figure 12. Example of reconstructions for an image from level 6 of the test dataset

Download: Full-size image PowerPoint slide

Figure 14. Example of reconstructions with the angular range of the levels 1, 4, and 7 (from left to right) for the image with easier holes (first row) and harder holes (second row)

Download: Full-size image PowerPoint slide

Figure 16. Example of reconstructions of an image from level 7 with different starting angles of the angular range

Download: Full-size image PowerPoint slide

Figure 18. Example of repeated reconstructions of images from levels 3 (first row) and 4 (second row)

Download: Full-size image PowerPoint slide

Table 1. Overview of each difficulty level

Difficulty level $ l $	1	2	3	4	5	6	7
Angular range	$ 90^\circ $	$ 80^\circ $	$ 70^\circ $	$ 60^\circ $	$ 50^\circ $	$ 40^\circ $	$ 30^\circ $
Angular increment	$ 0.5^\circ $	$ 0.5^\circ $	$ 0.5^\circ $	$ 0.5^\circ $	$ 0.5^\circ $	$ 0.5^\circ $	$ 0.5^\circ $
Number of projections $ N^l_p $	$ 181 $	$ 161 $	$ 141 $	$ 121 $	$ 101 $	$ 81 $	$ 61 $

| Show Table

DownLoad: CSV

Table 2. Results of MCC obtained for the reconstructions of each of the levels of the test dataset. Values presented in the format "Mean (standard deviation)"

Method / Level	1	2	3	4	5	6	7
DIP	0.987 (0.006)	0.987 (0.003)	0.963 (0.018)	0.937 (0.055)	0.936 (0.021)	0.731 (0.139)	0.653 (0.030)
DIP+MMP	0.988 (0.007)	0.987 (0.003)	0.967 (0.014)	0.936 (0.059)	0.951 (0.004)	0.765 (0.124)	0.680 (0.009)

| Show Table

DownLoad: CSV

Table 3. Results of MCC obtained for the reconstructions of images originally from level 1 (easier) and 7 (harder). The full-angle data of each image was used to generate limited-angle data with the angular range of each level

Difficulty / Level	1	2	3	4	5	6	7
Easier	0.993	0.990	0.988	0.988	0.987	0.984	0.970
Harder	0.932	0.927	0.843	0.810	0.741	0.700	0.671

| Show Table

DownLoad: CSV

Table 4. Neural network architecture used for the DIP

Parameter	DIP
Input size	$ 512 \times 512 \times 1 $
Output size	$ 512 \times 512 \times 1 $
No. of encoder layers	6
No. of decoder layers	6
No. of filters in each encoder layer	128 (all)
No. of filters in each decoder layer	128 (all)
Filters size in each encoder/decoder layer	$ 3 \times 3 $ (all)
No. of skip layers	6
No. of filters in each skip layer	12
Filters size in each skip layer	$ 1 \times 1 $
Activation function	Leaky ReLU
Activation function (last layer)	Sigmoid

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	J. Adler, H. Kohr and O. Öktem, Operator discretization library (ODL 0.7.0), January 2017, Last accessed 14, April 2023.
[2]	S. Arridge, P. Maass, O. Öktem and C.-B. Schönlieb, Solving inverse problems using data-driven models, Acta Numerica, 28 (2019), 1-174. doi: 10.1017/S0962492919000059.
[3]	R. C. Aster, B. Borchers and C. H. Thurber, Parameter Estimation and Inverse Problems, Elsevier, Amsterdam, Netherlands, 3^rd ed., 2018. doi: 10.1016/C2015-0-02458-3.
[4]	D. O. Baguer, J. Leuschner and M. Schmidt, Computed tomography reconstruction using deep image prior and learned reconstruction methods, Inverse Problems, 36 (2020), 094004. doi: 10.1088/1361-6420/aba415.
[5]	S. Barutcu, S. Aslan, A. K. Katsaggelos and D. Gürsoy, Limited-angle computed tomography with deep image and physics priors, Scientific Reports, 11 (2021), Article number: 17740. doi: 10.1038/s41598-021-97226-2.
[6]	G. Beylkin, Discrete radon transform, IEEE Transactions on Acoustics, Speech, and Signal Processing, 35 (1987), 162-172. doi: 10.1109/TASSP.1987.1165108.
[7]	M. Burger, B. Hahn and E. T. Quinto, Tomographic inverse problems: Theory and applications, Oberwolfach Reports, 16 (2020), 209-303. doi: 10.4171/owr/2019/4.
[8]	T. Buzug, Computed Tomography: From Photon Statistics to Modern Cone-beam CT, Springer, Berlin, 2008. ISBN: 978354039408-2.
[9]	T. Chan and L. Vese, An active contour model without edges, Scale-Space Theories in Computer Vision: Second International Conference. Springer, Greece, 1682 (2002), 141-151. doi: 10.1007/3-540-48236-9_13.
[10]	G. Chen, et al., Airnet: Fused analytical and iterative reconstruction with deep neural network regularization for sparse-data CT, Medical Physics (Lancaster), 47 (2020), 2916-2930. doi: 10.1002/mp.14170.
[11]	M. E. Davison, The ill-conditioned nature of the limited angle tomography problem, SIAM Journal on Applied Mathematics, 43 (1983), 428-448. doi: 10.1137/0143028.
[12]	J. T. Dobbins and H. P. McAdams, Chest tomosynthesis: Technical principles and clinical update, European Journal of Radiology, 72 (2009), 244-251. doi: 10.1016/j.ejrad.2009.05.054.
[13]	L. A. Feldkamp, L. C. Davis and J. W. Kress, Practical cone-beam algorithm, Journal of the Optical Society of America A, 1 (1984), 612-619. doi: 10.1364/JOSAA.1.000612.
[14]	J. Frikel and E. T. Quinto, Characterization and reduction of artifacts in limited angle tomography, Inverse Problems, 29 (2013), 125007. doi: 10.1088/0266-5611/29/12/125007.
[15]	J. Hadamard, Sur les problèmes aux dérivés partielles et leur signification physique, Princeton University Bulletin, 13 (1902), 49-52.
[16]	U. Hampel, Image reconstruction for hard field tomography, In Industrial Tomography, Woodhead Publishing Series in Electronic and Optical Materials, Woodhead Publishing, 2015,347-376. doi: 10.1016/B978-1-78242-118-4.00013-7.
[17]	P. Hansen, Discrete Inverse Problems: Insight and Algorithms, 1^st ed., Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 2010. doi: 10.1137/1.9780898718836.
[18]	K. Hämäläinen, et al., Sparse tomography, SIAM Journal on Scientific Computing, 35 (2013), B644-665. doi: 10.1137/120876277.
[19]	J. Hsieh, Computed Tomography, Press Monographs. SPIE Press, Bellingham, WA, 2^nd ed., July 2009. ISBN: 9780819475336.
[20]	U. Je, et al., Dental cone-beam CT reconstruction from limited-angle view data based on compressed-sensing (CS) theory for fast, low-dose X-ray imaging, Journal of the Korean Physical Society, 64 (2014), 1907-1911. doi: 10.3938/jkps.64.1907.
[21]	J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Springer, New York, 2005. doi: 10.1007/b138659.
[22]	S. Kida, et al., Cone beam computed tomography image quality improvement using a deep convolutional neural network, Cureus, 10 (2018), e2548. doi: 10.7759/cureus.2548.
[23]	T. Kluth, C. Bathke, M. Jiang and P. Maass, Joint super-resolution image reconstruction and parameter identification in imaging operator: analysis of bilinear operator equations, numerical solution, and application to magnetic particle imaging, Inverse Problems, 36 (2020), 124006. doi: 10.1088/1361-6420/abc2fe.
[24]	A. Kofler, et al., Neural networks-based regularization for large-scale medical image reconstruction, Physics in Medicine & Biology, 65 (2020), 135003. doi: 10.1088/1361-6560/ab990e.
[25]	H. Lan, J. Zhang, C. Yang and F. Gao, Compressed sensing for photoacoustic computed tomography based on an untrained neural network with a shape prior, Biomedical Optics Express, 12 (2021), 7835-7848. doi: 10.1364/boe.441901.
[26]	J. Lancaster and B. Hasegawa, Computed tomography, In Fundamental Mathematics and Physics of Medical Imaging, CRC Press, 2016,295-312. doi: 10.1201/9781315368214-28.
[27]	S. Latva-Äijö, et al., Helsinki tomography challenge 2022 (HTC 2022), Available at https://www.fips.fi/Helsinki_Tomography_Challenge_2022_v11.pdf, October 2022. Last accessed 14, April 2023.
[28]	J. Leuschner, et al., Quantitative comparison of deep learning-based image reconstruction methods for low-dose and sparse-angle CT applications, Journal of Imaging, 7 (2021), 44. doi: 10.3390/jimaging7030044.
[29]	H. Li, J. Schwab, S. Antholzer and M. Haltmeier, NETT: Solving inverse problems with deep neural networks, Inverse Problems, 36 (2020), 065005, 23 pp. doi: 10.1088/1361-6420/ab6d57.
[30]	L. Li, et al., Compressed sensing improved iterative reconstruction-reprojection algorithm for electron tomography, BMC Bioinformatics, 21 (2020), Article number: 202. doi: 10.1186/s12859-020-3529-3.
[31]	X. Li, G. Feng and J. Zhu, An algorithm of $\ell_1$-norm and $\ell_0$-norm regularization algorithm for CT image reconstruction from limited projection, International Journal of Biomedical Imaging, 2020 (2020), Article ID 8873865. doi: 10.1155/2020/8873865.
[32]	K. Lu, L. Ren and F.-F. Yin, A geometry-guided deep learning technique for CBCT reconstruction, Physics in Medicine & Biology, 66 (2021), 15LT01. doi: 10.1088/1361-6560/ac145b.
[33]	A. Majumdar, Compressed Sensing for Engineers, CRC Press, Taylor & Francis Group, Boca Raton, FL, 2019. ISBN: 9781032338712.
[34]	A. Meaney, F. S. Moura and S. Siltanen, Helsinki Tomography Challenge 2022 open tomographic dataset (HTC 2022), (2022). Available at: https://zenodo.org/record/7418878.
[35]	J. L. Mueller and S. Siltanen, Linear and Nonlinear Inverse Problems with Practical Applications, volume 10 of Computational Science Engineering, Society for Industrial and Applied Mathematics, United States, 2012.
[36]	F. Natterer, The Mathematics of Computerized Tomography, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2001. doi: 10.1137/1.9780898719284.
[37]	G. Ongie, et al., Deep learning techniques for inverse problems in imaging, IEEE Journal on Selected Areas in Information Theory, 1 (2020), 39-56. doi: 10.1109/JSAIT.2020.2991563.
[38]	N. Otsu, A threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics, 9 (1979), 62-66. doi: 10.1109/TSMC.1979.4310076.
[39]	P.-A. Poletti, et al., Low-dose versus standard-dose CT protocol in patients with clinically suspected renal colic, American Journal of Roentgenology, 188 (2007), 927-933. doi: 10.2214/ajr.06.0793.
[40]	A. Qayyum, et al., Untrained neural network priors for inverse imaging problems: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 (2023), 6511-6536. doi: 10.1109/tpami.2022.3204527.
[41]	E. T. Quinto, Limited-data tomography, In Computed Tomography: Algorithms, Insight, and Just Enough Theory, Society for Industrial and Applied Mathematics, 2021,123-153. doi: 10.1137/1.9781611976670.ch8.
[42]	H. Rabbani, N. Teyfouri and I. Jabbari, Low-dose cone-beam computed tomography reconstruction through a fast three-dimensional compressed sensing method based on the three-dimensional pseudo-polar Fourier transform, Journal of Medical Signals & Sensors, 12 (2022), 8.
[43]	S. Ravishankar, J. C. Ye and J. A. Fessler, Image reconstruction: From sparsity to data-adaptive methods and machine learning, Proceedings of the IEEE, 108 (2020), 86-109. doi: 10.1109/jproc.2019.2936204.
[44]	W. C. Scarfe and A. G. Farman, What is cone-beam CT and how does it work?, Dental Clinics of North America, 52 (2008), 707-730. doi: 10.1016/j.cden.2008.05.005.
[45]	J. K. Seo and E. J. Woo, Nonlinear Inverse Problems in Imaging, Wiley, Chichester, 2013. doi: 10.1002/9781118478141.
[46]	C. C. Shaw, Cone Beam Computed Tomography, CRC Press, Boca Raton, USA, 1^st ed., 2014. ISBN: 9781439846278.
[47]	L. A. Shepp and B. F. Logal, The Fourier reconstruction of a head section, IEEE Transactions on Nuclear Science, 21 (1974), 21-43. doi: 10.1109/TNS.1974.6499235.
[48]	J. H. Siewerdsen, Cone-beam CT systems, In Computed Tomography, Springer International Publishing, 2019, 11-26. doi: 10.1007/978-3-030-26957-9_2.
[49]	D. Tack, V. D. Maertelaer and P. A. Gevenois, Dose reduction in multidetector CT using attenuation-based online tube current modulation, American Journal of Roentgenology, 181 (2003), 331-334. doi: 10.2214/ajr.181.2.1810331.
[50]	N. Tirada, et al., Digital breast tomosynthesis: Physics, artifacts, and quality control considerations, RadioGraphics, 39 (2019), 413-426. doi: 10.1148/rg.2019180046.
[51]	D. Ulyanov, A. Vedaldi and V. Lempitsky, Deep image prior, International Journal of Computer Vision, 128 (2020), 1867-1888. doi: 10.1007/s11263-020-01303-4.
[52]	T. Wurfl, et al., Deep learning computed tomography: Learning projection-domain weights from image domain in limited angle problems, IEEE Transactions on Medical Imaging, 37 (2018), 1454-1463. doi: 10.1109/TMI.2018.2833499.
[53]	Y. Xie and Q. Li, A review of deep learning methods for compressed sensing image reconstruction and its medical applications, Electronics, 11 (2022), 586. doi: 10.3390/electronics11040586.
[54]	M. Zhang, S. Gu and Y. Shi, The use of deep learning methods in low-dose computed tomography image reconstruction: A systematic review, Complex & Intelligent Systems, 8 (2022), 5545-5561. doi: 10.1007/s40747-022-00724-7.