HD-DCDM: Hybrid-domain network for limited-angle computed tomography with deconvolution and conditional diffusion model

Jianyu Wang; Rongqian Wang; Lide Cai; Xintong Liu; Guochang Lin; Fukai Chen; Lingyun Qiu

doi:10.3934/ammc.2023008

Article Contents

2023, Volume 1, Issue 2: 202-218. Doi: 10.3934/ammc.2023008

This issue Previous Article Helsinki tomography challenge 2022: Description of the competition and dataset Next Article A deep learning approach using boundary shape information for limited-angle tomography reconstruction

HD-DCDM: Hybrid-domain network for limited-angle computed tomography with deconvolution and conditional diffusion model

1.
Yau Mathematical Sciences Center, Tsinghua University, China
2.
Department of Mathematical Sciences, Tsinghua University, China
3.
Yanqi Lake Beijing Institute of Mathematical Sciences and Applications, China

^*Corresponding author: Lingyun Qiu
^*Corresponding author: Lingyun Qiu

Jianyu Wang and Rongqian Wang contributed equally to this work

Received: July 2023

Revised: September 2023

Early access: October 2023

Published: December 2023

This work is supported by the National Key R&D Program of China (No. 2021YFA0719200) and the National Natural Science Foundation of China (No. 11971258, 61971292)

Abstract / Introduction Full Text(HTML) Figure(9) / Table(1) Related Papers Cited by

Abstract

Limited-angle computed tomography (LACT) has gained significant attention in recent years due to its wide range of applications. Despite the numerous algorithms proposed to improve imaging quality, reconstructing fine details remains a challenging problem. In this paper, we propose a novel hybrid domain framework that combines classical methods and learning-based methods to address this challenge. Our framework decomposes the solution of the least-squares problem into back-projection and deconvolution steps, leading to a significant improvement in reconstruction quality. Furthermore, we employ a conditional diffusion model to further fine-tune the reconstruction results, simultaneously preserving data consistency and enhancing the realness of the reconstructed images. The effectiveness of the proposed framework is evaluated using the Helsinki Tomography Challenge 2022 (HTC 2022) dataset. Comparative evaluations demonstrate that our framework outperforms previous methods in both visual quality and quantitative measures. These findings highlight the potential of the proposed framework in improving LACT reconstruction and offer valuable insights for advancing imaging techniques in various fields.

Keywords:

Mathematics Subject Classification: Primary: 45Q05, 68T07; Secondary: 68U10.

Citation:

FullText(HTML)

Figure 1. The flowchart of the HD-DCDM framework

Download: Full-size image PowerPoint slide

Figure 2. An illustration of the Res FFT-ReLU block

Download: Full-size image PowerPoint slide

Figure 3. An illustration of the workflow of the conditional diffusion model

Download: Full-size image PowerPoint slide

Figure 4. Typical examples of each class of training phantoms

Download: Full-size image PowerPoint slide

Figure 5. Reconstruction results from the HTC 2022 testing dataset for different methods with a 90-degree field of view. First column: reference images reconstructed by the FBP algorithm from full-sampled projections. Second column: images reconstructed by the FBP algorithm. Third column: images reconstructed by the TV method. Fourth column: images reconstructed by the FBPConvNet method. Fifth column: images reconstructed by the DIP method. Sixth column: images reconstructed by the HD-DCDM framework. The MCC scores are displayed below each image

Download: Full-size image PowerPoint slide

Figure 6. Reconstruction results from the HTC 2022 testing dataset for different methods with a 60-degree field of view. First column: reference images reconstructed by the FBP algorithm from full-sampled projections. Second column: images reconstructed by the FBP algorithm. Third column: images reconstructed by the TV method. Fourth column: images reconstructed by the FBPConvNet method. Fifth column: images reconstructed by the DIP method. Sixth column: images reconstructed by the HD-DCDM framework. The MCC scores are displayed below each image

Download: Full-size image PowerPoint slide

Figure 7. Reconstruction results from the HTC 2022 testing dataset for different methods with a 30-degree field of view. First column: reference images reconstructed by the FBP algorithm from full-sampled projections. Second column: images reconstructed by the FBP algorithm. Third column: images reconstructed by the TV method. Fourth column: images reconstructed by the FBPConvNet method. Fifth column: images reconstructed by the DIP method. Sixth column: images reconstructed by the HD-DCDM framework. The MCC scores are displayed below each image

Download: Full-size image PowerPoint slide

Figure 8. Ablation study. The reconstruction methods and corresponding MCC scores are presented above and below each image, respectively

Download: Full-size image PowerPoint slide

Figure 9. Comparisons of the outputs for the test images

Download: Full-size image PowerPoint slide

Table 1. The mean and variance value of one hundred repeated experiments with different test examples of level 7

	Mean	Variance
07a	0.72	$ 2.52\times 10^{-5} $
07b	0.80	$ 1.58\times 10^{-5} $
07c	0.78	$ 9.27 \times 10^{-5} $

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	L. Alves, R. Beraldo, R. Suyama, A.Takahata and J. Sims, Deep Image Prior with Sparsity Constraint for Limited-Angle Computed Tomography Reconstruction, 2023. Available from: https://github.com/robert-abc/HTC2022-Sparse-DIP. doi: 10.1109/CVPR.2018.00984.
[2]	R. Anirudh, H. Kim, J. Thiagarajan, K. Mohan, K. Champley and T. Bremer, Lose the views: Limited-Angle CT reconstruction via implicit Sinogram completion, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018, 6343-6352. doi: 10.1109/CVPR.2018.00664.
[3]	M. R. Banham and A. K. Katsaggelos, Digital image restoration, IEEE SIGNAL PROC. MAG., 14 (1997), 24-41. doi: 10.1109/79.581363.
[4]	R. Barbano, J. Leuschner, M. Schmidt, A. Denker, A. Hauptmann, P. Maass and B. Jin, An educated warm start for deep image prior-based micro CT reconstruction, IEEE Trans. Comput. Imaging, 8 (2022), 1210-1222. doi: 10.1109/TCI.2022.3233188.
[5]	A. Beck and M. Teboulle, Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems, IEEE Trans. Image Process., 18 (2009), 2419-2434. doi: 10.1109/TIP.2009.2028250.
[6]	A. Biguri, M. Dosanjh, S. Hancock and M. Soleimani, TIGRE: A MATLAB-GPU toolbox for CBCT image reconstruction, Biomed. Phys. Eng. Express, 2 (2016), 055010. doi: 10.1088/2057-1976/2/5/055010.
[7]	T. A. Bubba, M. Galinier, M. Lassas, M. Prato, L. Ratti and S. Siltanen, Deep neural networks for inverse problems with pseudodifferential operators: An application to limited-angle tomography, SIAM J. Imaging Sci., 14 (2021), 470-505. doi: 10.1137/20M1343075.
[8]	T. Bubba, G. Kutyniok, M. Lassas, M. März, W. Samek, S. Siltanen and V. Srinivasan, Learning the invisible: A hybrid deep learning-shearlet framework for limited-angle computed tomography, Inverse Probl., 35, (2019), 064002, 38 pp. doi: 10.1088/1361-6420/ab10ca.
[9]	C. Chen, Y. Xing, H. Gao, L. Zhang and Z. Chen, Sam's net: A self-augmented multistage deep-learning network for end-to-end reconstruction of limited-angle CT, IEEE Trans. Med. Imaging, 41 (2022), 2912-2924. doi: 10.1109/TMI.2022.3175529.
[10]	Z. Chen, X. Jin, L. Li and G. Wang, A limited-angle CT reconstruction method based on anisotropic TV minimization, Phys. Med. Biol., 58 (2013), 2119. doi: 10.1088/0031-9155/58/7/2119.
[11]	D. Chicco and G. Jurman, The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation, BMC Genom., 21 (2020), 1-13. doi: 10.1186/s12864-019-6413-7.
[12]	X. Deng, X. Zhao, M. Li and H. Li, Limited-angle CT reconstruction with generalized shrinkage operators as regularizers, Inverse Probl. Imaging, 15 (2021), 1287-1306. doi: 10.3934/ipi.2021019.
[13]	Y. Ge, Q. Zhang, Z. Hu, J. Chen, W. Shi, H. Zheng and D. Liang, Deconvolution-Based Backproject-Filter (BPF) computed tomography image reconstruction method using deep learning technique, preprint, 2018. arXiv: 1807.01833.
[14]	J. Ho, A. Jain and P. Abbeel, Denoising diffusion probabilistic models, NIPS, 33 (2020), 6840-6851.
[15]	D. Hu, Y. Zhang, G. Quan, J. Xiang, G. Coatrieux and S. Luo, et al., CROSS: Cross-domain residual-optimization based structure strengthening reconstruction for limited-angle CT, IEEE Transactions on Radiation and Plasma Medical Sciences, 7 (2023), 521-531. doi: 10.1109/TRPMS.2023.3242662.
[16]	K. Jin, M. McCann, E. Froustey and M. Unser, Deep convolutional neural network for inverse problems in imaging, IEEE Trans. Image Process., 26 (2017), 4509-4522. doi: 10.1109/TIP.2017.2713099.
[17]	D. Kingma and J. Ba, Adam: A method for stochastic optimization, preprint, 2014. arXiv: 1412.6980.
[18]	E. Lewis and W. Miller, Computational Methods of Neutron Transport, 1$^{st}$ edition, Wiley-Interscience, Hoboken, 184.
[19]	Z. Li, A. Cai, L. Wang, W. Zhang, C. Tang and L. Li, et al., Promising generative adversarial network based sinogram inpainting method for ultra-limited-angle computed tomography imaging, Sensors, 19 (2019), 3941. doi: 10.3390/s19183941.
[20]	Z. Li, W. Zhang, L. Wang, A. Cai, N. Liang, B. Yan and L. Li, A sinogram inpainting method based on generative adversarial network for limited-angle computed tomography, in Proceedings of the 15th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine, Philadelphia, PA, USA, 2019, 345-349. doi: 10.1117/12.2533757.
[21]	I. Loshchilov and F. Hutter, SGDR: Stochastic gradient descent with warm restarts, preprint, 2016. arXiv: 1608.03983.
[22]	C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li and J. Zhu, DPM-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps, Advances in Neural Information Processing Systems, 35 (2022), 5775-5787.
[23]	X. Mao, Y. Liu, F. Liu, Q. Li, W. Shen and Y. Wang, Intriguing findings of frequency selection for image deblurring, Proceedings of the AAAI Conference on Artificial Intelligence, 37 (2023), 1905-1913. doi: 10.1609/aaai.v37i2.25281.
[24]	S. Nah, T. Kim and K. Lee, Deep multi-scale convolutional neural network for dynamic scene deblurring, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 2017, 257-265. doi: 10.1109/CVPR.2017.35.
[25]	F. Natterer, Mathematics for Computerized Tomography, 1$^{st}$ edition, SIAM, Philadelphia, 2001. doi: 10.1137/1.9780898719284.
[26]	E. Quinto, An introduction to X-ray tomography and radon transforms, Proceedings of Symposia in Applied Mathematics, Providence, USA, 63 (2006), 1-23. doi: 10.1090/psapm/063/2208234.
[27]	E. Sidky, C. Kao and X. Pan, Accurate image reconstruction from few-views and limited-angle data in divergent-beam CT, J. X-Ray. Sci. Technol., 14 (2006), 119-139.
[28]	J. Song, C. Meng and S. Ermon, Denoising diffusion implicit models, preprint, 2020. arXiv: 2010.02502.
[29]	Y. Sun and J. Tao, Image reconstruction from few views by $\ell_0$-norm optimization, Chin. Phys. B., 23 (2014), 078703. doi: 10.1088/1674-1056/23/7/078703.
[30]	R. Tovey, M. Benning, C. Brune, M. J. Lagerwerf, S. M. Collins, R. K. Leary, P. A. Midgley and C.-B. Schönlieb, Directional sinogram inpainting for limited-angle tomography, Inverse Probl., 35 (2019), 024004, 29 pp. doi: 10.1088/1361-6420/aaf2fe.
[31]	D. Ulyanov, A. Vedaldi and V. Lempitsky, Deep image prior, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018, 9446-9454.
[32]	M. U. Ghani and W. C. Karl, Integrating data and image domain deep learning for limited-angle tomography using consensus equilibrium, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea, 2019, 3922-3932. doi: 10.1109/ICCVW.2019.00486.
[33]	W. van Aarle, W. J. Palenstijn, J. Cant, E. Janssens, F. Bleichrodt, A. Dabravolski, J. De Beenhouwer, K. J. Batenburg and J. Sijbers, Fast and flexible X-ray tomography using the ASTRA toolbox, Opt. Express, 24 (2016), 25129-25147. doi: 10.1364/OE.24.025129.
[34]	W. van Aarle, W. J. Palenstijn, J. De Beenhouwer, T. Altantzis, S. Bals, K. J. Batenburg and J. Sijbers, The ASTRA Toolbox: A platform for advanced algorithm development in electron tomography, Ultramicroscopy, 157 (2015), 35-47. doi: 10.1016/j.ultramic.2015.05.002.
[35]	C. Wang and L. Zeng, Error bounds and stability in the $\ell_0$ regularized for CT reconstruction from small projections, Inverse Probl. Imaging, 10 (2016), 829-853. doi: 10.3934/ipi.2016023.
[36]	C. Wang, H. Zhang, Q. Li, K. Shang, Y. Lyu, B. Dong and S. Zhou, Improving generalizability in limited-angle CT reconstruction with sinogram extrapolation, Proceedings of the 24th International Conference, Strasbourg, France, 2021, 56-96. doi: 10.1007/978-3-030-87231-1_9.
[37]	S. Wang, H. Yu, Y. Xi, C. Gong, W. Wu and F. Liu, Spectral-image decomposition with energy-fusion sensing for spectral CT reconstruction, IEEE Transactions on Instrumentation and Measurement, 70 (2021), 1-11. doi: 10.1109/TIM.2021.3078555.
[38]	T. Wang, K. Nakamoto, H. Zhang and H. Liu, Reweighted anisotropic total variation minimization for limited-angle CT reconstruction, IEEE. T. Nucl. Sci., 64 (2017), 2742-2760. doi: 10.1109/TNS.2017.2750199.
[39]	Y. Wang, T. Yang and W. Huang, Limited-angle computed tomography reconstruction using combined FDK-based neural network and U-Net, in Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, Canada, 2020, 1572-1575. doi: 10.1109/EMBC44109.2020.9176040.
[40]	Y. Wang and J. Yu, Zero-shot image restoration using denoising diffusion null-space model, preprint, 2022. arXiv: 2212.00490.
[41]	T. Würfl, M. Hoffmann, V. Christlein, K. Breininger, Y. Huang, M. Unberath and A. Maier, Deep learning computed tomography: Learning projection-domain weights from image domain in limited-angle problems, IEEE Trans. Med. Imaging, 37, (2018), 1454-1463. doi: 10.1109/TMI.2018.2833499.
[42]	J. Xu, Y. Zhao, H. Li and P. Zhang, An image reconstruction model regularized by edge-preserving diffusion and smoothing for limited-angle computed tomography, Inverse Probl., 35 (2019), 085004, 34 pp. doi: 10.1088/1361-6420/ab08f9.
[43]	M. Xu, D. Hu, F. Luo, F. Liu, S. Wang and W. Wu, Limited-angle X-ray CT reconstruction using image gradient $\ell_0$-norm with dictionary learning, IEEE Transactions on Radiation and Plasma Medical Sciences, 5 (2021), 78-87. doi: 10.1109/TRPMS.2020.2991887.
[44]	G. Zeng, Medical Image Reconstruction: A Conceptual Tutorial, 1$^{st}$ edition, Springer, New York, 2010. doi: 10.1007/978-3-642-05368-9.
[45]	Q. Zhang, Z. Hu, C. Jiang, H. Zheng, Y. Ge and D. Liang, Artifact removal using a hybrid-domain convolutional neural network for limited-angle computed tomography imaging, Phys. Med. Biol., 65 (2020), 155010. doi: 10.1088/1361-6560/ab9066.
[46]	Y. Zhang, D. Hu, Z. Yan, Q. Zhao, G. Quan and S. Luo, et al., TIME-Net: Transformer-integrated multi-encoder network for limited-angle artifact removal in dual-energy CBCT, Med Image Anal., 83 (2023), 102650. doi: 10.1016/j.media.2022.102650.
[47]	B. Zhou, S. K. Zhou, J. S. Duncan and C. Liu, Limited view tomographic reconstruction using a cascaded residual dense spatial-channel attention network with projection data fidelity layer, IEEE Trans. Med. Imaging, 40, (2021), 1792-1804. doi: 10.1109/TMI.2021.3066318.