A deep learning approach using boundary shape information for limited-angle tomography reconstruction

Chao Wang; Ji Li

doi:10.3934/ammc.2023012

Article Contents

2023, Volume 1, Issue 2: 219-231. Doi: 10.3934/ammc.2023012

This issue Previous Article HD-DCDM: Hybrid-domain network for limited-angle computed tomography with deconvolution and conditional diffusion model Next Article Preface

A deep learning approach using boundary shape information for limited-angle tomography reconstruction

Chao Wang^1, , and
Ji Li^2,

1.
Department of Statistics and Data Science, National University of Singapore, Singapore
2.
Academy for Multidisciplinary Studies, Capital Normal University, Beijing, China

^*Corresponding author: Chao Wang
^*Corresponding author: Chao Wang

Received: June 2023

Revised: November 2023

Early access: December 13, 2023

Published online: December 13, 2023

Abstract / Introduction Full Text(HTML) Figure(5) / Table(1) Related Papers Cited by

Abstract

In this paper, we focused on the deep learning solution to the severely ill-posed inverse problem of limited angle computed tomography. The limited measurement data does not hold all the information of the imaging object to enable high-precision inversion. In this scenario, as the analytical filtered back-projection (FBP) method inevitably introduced artifacts in the reconstruction, a variational approach was more suitable to solve such a problem, where prior information was incorporated as a regularization term. Though the traditional prior from the handcrafted sparsity promoting regularization has been widely used in the past decade, these regularization-based approaches do not perform well for the extremely hard cases of missing many angle views in the sinogram. To address the issue of hard cases, deep learning was leveraged in this paper to learn the solution map from the sinogram to the reconstruction. To reduce the computational demand, instead of the optimization-unrolling-based network architecture, we adopted the FBP plus U-Net refinement framework, where the FBP reconstruction serves as the initial reconstruction and the input to the refinement network. The prerequisite of the training dataset is the main challenge for the Helsinki Tomography Challenge 2022 (HTC2022) task since only the full-angle sinograms of five phantoms are provided. Thus, we roll the sinogram and rotate the provided ground truth to augment the dataset for each level of the number of observed angles. To better reconstruct the object, we fed the information of the outer circle boundary into the network. Although the test performance shows that our approach is a competitive method, yet not the leading solution in the competition, our idea can be incorporated into other award-winning solutions to improve their performance.

Keywords:

Mathematics Subject Classification: Primary: 68T07, 68U10; Secondary: 94A08.

Citation:

FullText(HTML)

Figure 1. The total five phantoms used in the training dataset

Download: Full-size image PowerPoint slide

Figure 2. Comparison of reconstructions of the phantom from our built test dataset with $ 70^\circ $ angles. The first row is the reconstruction with the shape prior; the second row is the one without the shape prior. The third row is the one with combined training loss and the shape prior. The fourth row is the one from the learned primal-dual algorithm [1]

Download: Full-size image PowerPoint slide

Figure 3. Comparison of reconstructions of the phantom from the testing dataset with $ 70^\circ $ angles. The first row is the reconstruction with the shape prior. The second row is the one with combined training loss and the shape prior. The third row is the one from the learned primal-dual algorithm [1]

Download: Full-size image PowerPoint slide

Figure 4. Reconstruction of the phantoms for seven levels. The (a), (c) and (e) columns are the truth and the (b), (d) and (f) columns are the reconstruction

Download: Full-size image PowerPoint slide

Figure 5. Generated patterns with rectangle holes

Download: Full-size image PowerPoint slide

Table 1. Scores of reconstruction results of samples in the test dataset

Sample	Level 1	Level 2	Level 3	Level 4	Level 5	Level 6	Level 7
A	0.98991	0.86566	0.59670	0.74181	0.96497	0.85329	0.58669
B	0.98584	0.84429	0.50683	0.64274	0.68449	0.67221	0.75349
C	0.98032	0.70310	0.96522	0.65008	0.92236	0.56497	0.61420

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	J. Adler and O. Öktem, Learned primal-dual reconstruction, IEEE Transactions on Medical Imaging, 37 (2018), 1322-1332. doi: 10.1109/TMI.2018.2799231.
[2]	R. Anirudh, H. Kim, J. J. Thiagarajan, K. A. Mohan, K. Champley and T. Bremer, Lose the views: limited angle CT reconstruction via implicit sinogram completion, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, 6343-6352. doi: 10.1109/CVPR.2018.00664.
[3]	E. J. Candes and D. L. Donoho, Curvelets and reconstruction of images from noisy radon data, in Wavelet Applications in Signal and Image Processing VIII, vol. 4119, SPIE, 2000,108-117. doi: 10.1117/12.408569.
[4]	J. Frikel, Sparse regularization in limited angle tomography, Applied and Computational Harmonic Analysis, 34 (2013), 117-141. doi: 10.1016/j.acha.2012.03.005.
[5]	J. Gu and J. C. Ye, Multi-scale wavelet domain residual learning for limited-angle ct reconstruction, in Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine, Fully3D conference organization, 2017.
[6]	K. Hammernik, T. Würfl, T. Pock and A. Maier, A deep learning architecture for limited-angle computed tomography reconstruction, in Bildverarbeitung für die Medizin 2017: Algorithmen-Systeme-Anwendungen. Proceedings des Workshops vom 12. bis 14. März 2017 in Heidelberg, Springer, 2017, 92-97. doi: 10.1007/978-3-662-54345-0_25.
[7]	J. S. Jørgensen, S. B. Coban, W. R. Lionheart, S. A. McDonald and P. J. Withers, Sparsebeads data: Benchmarking sparsity-regularized computed tomography, Measurement Science and Technology, 28 (2017), 124005. doi: 10.1088/1361-6501/aa8c29.
[8]	A. C. Kak and M. Slaney, Principles of Computerized Tomographic Imaging, SIAM, 2001. doi: 10.1137/1.9780898719277.
[9]	K. Kim, J. C. Ye, W. Worstell, J. Ouyang, Y. Rakvongthai, G. El Fakhri and Q. Li, Sparse-view spectral ct reconstruction using spectral patch-based low-rank penalty, IEEE Transactions on Medical Imaging, 34 (2014), 748-760. doi: 10.1109/TMI.2014.2380993.
[10]	E. Klann, E. T. Quinto and R. Ramlau, Wavelet methods for a weighted sparsity penalty for region of interest tomography, Inverse Problems, 31 (2015), 025001. doi: 10.1088/0266-5611/31/2/025001.
[11]	Y. Li, K. Li, C. Zhang, J. Montoya and G.-H. Chen, Learning to reconstruct computed tomography images directly from sinogram data under a variety of data acquisition conditions, IEEE Transactions on Medical Imaging, 38 (2019), 2469-2481. doi: 10.1109/TMI.2019.2910760.
[12]	G. Ongie, A. Jalal, C. A. Metzler, R. G. Baraniuk, A. G. Dimakis and R. Willett, Deep learning techniques for inverse problems in imaging, Ieee Journal on Selected Areas in Information Theory, 1 (2020), 39-56. doi: 10.1109/JSAIT.2020.2991563.
[13]	M. Rantala, S. Vanska, S. Jarvenpaa, M. Kalke, M. Lassas, J. Moberg and S. Siltanen, Wavelet-based reconstruction for limited-angle X-ray tomography, IEEE Transactions on Medical Imaging, 25 (2006), 210-217. doi: 10.1109/TMI.2005.862206.
[14]	Y. Romano, M. Elad and P. Milanfar, The little engine that could: regularization by denoising (RED), SIAM Journal on Imaging Sciences, 10 (2017), 1804-1844. doi: 10.1137/16M1102884.
[15]	O. Ronneberger, P. Fischer and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, Springer, 2015,234-241. doi: 10.1007/978-3-319-24574-4_28.
[16]	O. Ronneberger, P. Fischer and T. Brox, Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015 Conference Proceedings, 2022. doi: 10.1007/978-3-319-24574-4_28.
[17]	L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D: Nonlinear Phenomena, 60 (1992), 259-268. doi: 10.1016/0167-2789(92)90242-F.
[18]	E. Ryu, J. Liu, S. Wang, X. Chen, Z. Wang and W. Yin, Plug-and-play methods provably converge with properly trained denoisers, in International Conference on Machine Learning, PMLR, 2019, 5546-5557.
[19]	L. Shen, J. Pauly and L. Xing, Nerp: Implicit neural representation learning with prior embedding for sparsely sampled image reconstruction, IEEE Transactions on Neural Networks and Learning Systems. doi: 10.1109/TNNLS.2022.3177134.
[20]	T. Shen, X. Li, Z. Zhong, J. Wu and Z. Lin, R^{^} 2 2-net: Recurrent and recursive network for sparse-view CT artifacts removal, in Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part VI 22, Springer, 2019,319-327. doi: 10.1007/978-3-030-32226-7_36.
[21]	E. Y. Sidky, C.-M. Kao and X. Pan, Accurate image reconstruction from few-views and limited-angle data in divergent-beam ct, Journal of X-ray Science and Technology, 14 (2006), 119-139.
[22]	E. Y. Sidky and X. Pan, Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization, Physics in Medicine & Biology, 53 (2008), 4777. doi: 10.1088/0031-9155/53/17/021.
[23]	Y. Sun, J. Liu, M. Xie, B. Wohlberg and U. S. Kamilov, Coil: Coordinate-based internal learning for tomographic imaging, IEEE Transactions on Computational Imaging, 7 (2021), 1400-1412. doi: 10.1109/TCI.2021.3094062.
[24]	D. Ulyanov, A. Vedaldi and V. Lempitsky, Instance normalization: The missing ingredient for fast stylization, arXiv preprint, arXiv: 1607.08022.
[25]	D. Ulyanov, A. Vedaldi and V. Lempitsky, Deep image prior, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, 9446-9454. doi: 10.1007/s11263-020-01303-4.
[26]	S. V. Venkatakrishnan, C. A. Bouman and B. Wohlberg, Plug-and-play priors for model based reconstruction, in 2013 IEEE Global Conference on Signal and Information Processing, IEEE, 2013,945-948. doi: 10.1109/GlobalSIP.2013.6737048.
[27]	G. Zang, R. Idoughi, R. Li, P. Wonka and W. Heidrich, Intratomo: self-supervised learning-based tomography via sinogram synthesis and prediction, in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, 1960-1970. doi: 10.1109/ICCV48922.2021.00197.
[28]	H. Zhang, L. Li, K. Qiao, L. Wang, B. Yan, L. Li and G. Hu, Image prediction for limited-angle tomography via deep learning with convolutional neural network, arXiv preprint, arXiv: 1607.08707.
[29]	M. Zhang, S. Gu and Y. Shi, The use of deep learning methods in low-dose computed tomography image reconstruction: A systematic review, Complex & Intelligent Systems, 8 (2022), 5545-5561. doi: 10.1007/s40747-022-00724-7.
[30]	B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Rosen and M. S. Rosen, Image reconstruction by domain-transform manifold learning, Nature, 555 (2018), 487-492. doi: 10.1038/nature25988.