Learnable Douglas-Rachford iteration and its applications in DOT imaging

Jiulong Liu; Nanguang Chen; Hui Ji

doi:10.3934/ipi.2020031

Article Contents

2020, Volume 14, Issue 4: 683-700. Doi: 10.3934/ipi.2020031

This issue Previous Article Thermoacoustic Tomography with circular integrating detectors and variable wave speed Next Article Reconstruction of the derivative of the conductivity at the boundary

Learnable Douglas-Rachford iteration and its applications in DOT imaging

1.
Department of Mathematics, National University of Singapore, Singapore 119076
2.
Department of Biomedical Engineering, National University of Singapore, Singapore 117583

^* Corresponding author: Jiulong Liu
^* Corresponding author: Jiulong Liu

Received: July 2019

Revised: January 2020

Published: May 2020

Abstract / Introduction Full Text(HTML) Figure(9) / Table(3) Related Papers Cited by

Abstract

How to overcome the ill-posed nature of inverse problems is a pervasive problem in medical imaging. Most existing solutions are based on regularization techniques. This paper proposed a deep neural network (DNN) based image reconstruction method, the so-called DR-Net, that leverages the interpretability of existing regularization methods and adaptive modeling capacity of DNN. Motivated by a Douglas-Rachford fixed-point iteration for solving $ \ell_1 $-norm relating regularization model, the proposed DR-Net learns the prior of the solution via a U-Net based network, as well as other important regularization parameters. The DR-Net is applied to solve image reconstruction problem in diffusion optical tomography (DOT), a non-invasive imaging technique with many applications in medical imaging. The experiments on both simulated and experimental data showed that the proposed DNN based image reconstruction method significantly outperforms existing regularization methods.

Keywords:

Mathematics Subject Classification: Primary: 94A08, 68U10; Secondary: 92B20.

Citation:

Full Text(HTML)

Figure 1. A prototype time-resolved diffuse optical tomography system designed for optical imaging of human breast [24]

Download: Full-size image PowerPoint slide

Figure 2. X-Net: architecture of components of DR-Net, including $ \mathcal{D_\theta} $, $ \Phi_\vartheta $ and $ \mathcal{D_\theta}^\top $. Note that the weights of $ \mathcal{D_\theta} $ and $ \mathcal{D_\theta}^\top $ can be shared with each other, or learned individually.

Download: Full-size image PowerPoint slide

Figure 3. Phantom shapes for simulated data, each cubic container is of size $ 5mm \times 5mm\times 5mm $ ($ 1 \times 1 \times 1 $ voxel)

Download: Full-size image PowerPoint slide

Figure 4. Reconstructed absorption coefficients from simulated measurements with phantom size of $ 10mm \times 10mm \times 5mm $ in depth of $ 30mm $

Download: Full-size image PowerPoint slide

Figure 5. Reconstructed scattering coefficients from simulated measurements with phantom of $ 10mm \times 10mm \times 5mm $ in depth of $ 30mm $

Download: Full-size image PowerPoint slide

Figure 6. Reconstructed absorption coefficients from experimental measurements $ \{I_{15}, I_{16}, I_{17}\} $ with phantom of $ 5 mm \times 10mm \times 5mm $ in depth of $ 25mm $

Download: Full-size image PowerPoint slide

Figure 7. Reconstructed scattering coefficients from experimental measurements $ \{I_{15}, I_{16}, I_{17}\} $ with phantom of $ 5 mm \times 10mm \times 5mm $ in depth of $ 25mm $

Download: Full-size image PowerPoint slide

Figure 8. Outputs of inversion blocks for absorption coefficients $ u_a^k $ of all stages in inference phase from simulated measurements with phantom size of $ 10mm \times 10mm \times 5mm $ in depth of $ 15mm $

Download: Full-size image PowerPoint slide

Figure 9. Outputs of de-artifacting blocks for absorption coefficients $ \mathcal{D}_{\theta^\ast_k}^\top v_a^k $ of all stages in inference phase from simulated measurements with phantom size of $ 10mm \times 10mm \times 5mm $ in depth of $ 15mm $

Download: Full-size image PowerPoint slide

Table 1. Experimental dataset ($ Q\neq \phi $)

Depth(mm)	5	15	25	35	45
raw dataset	$ T_1 = \{I_{i}\}_{1}^5 $	$ T_2 = \{I_{i}\}_{6}^{13} $	$ T_3 = \{I_{i}\}_{14}^{18} $	$ T_4 = \{I_{i}\}_{15}^{19} $	$ T_5 = \{I_{i}\}_{20}^{24} $
Augmentation	$ Q\subset T_1 $	$ Q\subset T_2 $	$ Q\subset T_3 $	$ Q\subset T_4 $	$ Q\subset T_5 $
Data size	31	255	31	31	31
Purpose	Training	Training	Testing	Training	Training

| Show Table

DownLoad: CSV

Table 2. CNR of reconstructed phantom in Fig. 4-7 from simulated data and experimental data

Data	Results	Pixel	Tikhonov	TV	Post-net	Learned PD	DR-Net
sim	$ u_a $	1	1.04	1.33	13.53	83.65	125.94
		2	1.46	1.28	11.27	77.82	158.06
		3	1.58	1.42	16.02	66.74	100.64
		4	1.94	2.17	15.58	75.16	218.52
	$ u_s $	1	1.48	2.33	6.06	84.81	187.18
		2	1.94	2.32	5.61	77.82	123.83
		3	1.18	1.17	4.83	66.74	51.55
		4	1.36	2.51	5.61	75.16	34.63
exp	$ u_a $	1	1.75	1.12	3.14	0.92	5.09
	$ u_a $	2	2.21	2.39	1.70	4.93	15.57
	$ u_s $	1	1.38	0.64	1.16	0.96	7.09
	$ u_s $	2	2.40	1.55	2.21	6.7607	18.98

| Show Table

DownLoad: CSV

Table 3. Averaged PSNR and SSIM of reconstructed images from simulated data and experimental data

Data	Results	Measure	Tikhonov	TV	Post-net	Learned PD	DR-Net
sim	$ u_a $	PSNR	31.68	31.91	38.22	39.46	$ {\mathbf 40.98} $
	$ u_a $	SSIM	0.9182	0.9275	0.9759	0.9807	$ {\mathbf 0.9881} $
	$ u_s $	PSNR	31.29	32.04	34.56	$ {\mathbf 41.34} $	38.03
	$ u_s $	SSIM	0.9301	0.9401	0.9740	$ {\mathbf 0.9914 } $	0.9898
exp	$ u_a $	PSNR	28.16	28.36	29.01	28.07	$ {\mathbf 29.33 } $
	$ u_a $	SSIM	0.8612	0.8871	0.9441	0.9412	$ {\mathbf 0.9460 } $
	$ u_s $	PSNR	28.92	29.20	29.94	29.46	$ {\mathbf 31.13 } $
	$ u_s $	SSIM	0.8870	0.9239	0.9587	0.9700	$ {\mathbf 0.9779} $

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	J. Adler and O. Öktem, Learned primal-dual reconstruction, IEEE Transactions on Medical Imaging, 37 (2018), 1322-1332. doi: 10.1109/TMI.2018.2799231.
[2]	J. Adler and O. Öktem, Solving ill-posed inverse problems using iterative deep neural networks, Inverse Problems, 33 (2017), 124007, 24 pp. doi: 10.1088/1361-6420/aa9581.
[3]	S. Boyd, N. Parikh, E. Chu, B. Peleato and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends® in Machine Learning, 3 (2011), 1–122.
[4]	J.-F. Cai, B. Dong, S. Osher and Z. Shen, Image restoration: Total variation, wavelet frames, and beyond, J. Amer. Math. Soc., 25 (2012), 1033-1089. doi: 10.1090/S0894-0347-2012-00740-1.
[5]	J.-F. Cai, H. Ji, Z. Shen and G.-B. Ye, Data-driven tight frame construction and image denoising, Appl. Comput. Harmon. Anal., 37 (2014), 89-105. doi: 10.1016/j.acha.2013.10.001.
[6]	N. Cao, A. Nehorai and M. Jacob, Image reconstruction for diffuse optical tomography using sparsity regularization and expectation-maximization algorithm, Optics Express, 15 (2007), 13695-13708. doi: 10.1364/OE.15.013695.
[7]	K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, Image denoising with block-matching and 3d filtering, in Image Processing: Algorithms and Systems, Neural Networks, and Machine Learning, 6064, Proc. SPIE, 2006, 606414. doi: 10.1117/12.643267.
[8]	D. Davis and W. Yin, A three-operator splitting scheme and its optimization applications, Set-Valued Var. Anal., 25 (2017), 829-858. doi: 10.1007/s11228-017-0421-z.
[9]	T. Goldstein and S. Osher, The split Bregman method for $L1$-regularized problems, SIAM J. Imaging Sci., 2 (2009), 323-343. doi: 10.1137/080725891.
[10]	I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., Generative adversarial nets, in Advances in Neural Information Processing Systems, 2014, 2672–2680.
[11]	I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin and A. Courville, Improved training of Wasserstein GANs, preprint, arXiv: 1704.00028.
[12]	K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock and F. Knoll, Learning a variational network for reconstruction of accelerated MRI data, Magnetic Resonance in Medicine, 79 (2018), 3055-3071. doi: 10.1007/978-3-319-66709-6.
[13]	K. H. Jin, M. T. McCann, E. Froustey and M. Unser, Deep convolutional neural network for inverse problems in imaging, IEEE Trans. Image Process., 26 (2017), 4509-4522. doi: 10.1109/TIP.2017.2713099.
[14]	E. Kang, J. Min and J. C. Ye, A deep convolutional neural network using directional wavelets for low-dose x-ray CT reconstruction, Medical Physics, 44 (2017), e360–e375. doi: 10.1002/mp.12344.
[15]	V. Kolehmainen, M. Vauhkonen, J. P. Kaipio and S. R. Arridge, Recovery of piecewise constant coefficients in optical diffusion tomography, Optics Express, 7 (2000), 468-480. doi: 10.1364/OE.7.000468.
[16]	C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, et al., Photo-realistic single image super-resolution using a generative adversarial network, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 105–114. doi: 10.1109/CVPR.2017.19.
[17]	P.-L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16 (1979), 964-979. doi: 10.1137/0716071.
[18]	J. Liu, Y. Hu, J. Yang, Y. Chen, H. Shu, L. Luo, et al., 3d feature constrained reconstruction for low-dose CT imaging, IEEE Transactions on Circuits and Systems for Video Technology, 28 (2018), 1232–1247. doi: 10.1109/TCSVT.2016.2643009.
[19]	J. Liu, A. I. Aviles-Rivero, H. Ji and C.-B. Schönlieb, Rethinking medical image reconstruction via shape prior, going deeper and faster: Deep joint indirect registration and reconstruction, preprint, arXiv: 1912.07648. doi: 10.1016/j.mathsocsci.2015.03.001.
[20]	J. Liu, H. Ding, S. Molloi, X. Zhang and H. Gao, TICMR: Total image constrained material reconstruction via nonlocal total variation regularization for spectral CT, IEEE Transactions on Medical Imaging, 35 (2016), 2578-2586. doi: 10.1109/TMI.2016.2587661.
[21]	J. Liu, T. Kuang and X. Zhang, Image reconstruction by splitting deep learning regularization from iterative inversion, in Medical Image Computing and Computer Assisted Intervention, 11070, Lecture Notes in Computer Science, Springer, Cham, 2018,224–231. doi: 10.1007/978-3-030-00928-1_26.
[22]	T. Liu, M. Gong and D. Tao, Large-cone nonnegative matrix factorization, IEEE Trans. Neural Netw. Learn. Syst., 28 (2017), 2129-2142. doi: 10.1109/tnnls.2016.2514360.
[23]	T. Meinhardt, M. Moller, C. Hazirbas and D. Cremers, Learning proximal operators: Using denoising networks for regularizing inverse imaging problems, in IEEE International Conference on Computer Vision (ICCV), 2017, 1781–1790. doi: 10.1109/ICCV.2017.198.
[24]	W. Mo and N. Chen, Design of an advanced time-domain diffuse optical tomography system, IEEE Journal of Selected Topics in Quantum Electronics, 16 (2010), 581-587.
[25]	S. Nowozin, B. Cseke and R. Tomioka, f-GAN: Training generative neural samplers using variational divergence minimization, in Advances in Neural Information Processing Systems, 2016,271–279.
[26]	O. Ronneberger, P. Fischer and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, 9351, Lecture Notes in Computer Science, Springer, Cham, 2015,234–241. doi: 10.1007/978-3-319-24574-4_28.
[27]	L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), 259-268. doi: 10.1016/0167-2789(92)90242-F.
[28]	J. Sun, H. Li, Z. Xu and Y. Yang, Deep ADMM-Net for compressive sensing MRI, in Advances in Neural Information Processing Systems, 2016, 10–18.
[29]	A. N. Tihonov, Solution of incorrectly formulated problems and the regularization method, Soviet Math. Dokl., 4 (1963), 1035-1038.
[30]	P. Vincent, H. Larochelle, Y. Bengio and P.-A. Manzagol, Extracting and composing robust features with denoising autoencoders, in Proceedings of the 25th International Conference on Machine learning, ACM, 2008, 1096–1103. doi: 10.1145/1390156.1390294.
[31]	J. Xie, L. Xu and E. Chen, Image denoising and inpainting with deep neural networks, in Advances in Neural Information Processing Systems, 2012,341–349.
[32]	X. Yang, R. Kwitt, M. Styner and M. Niethammer, Quicksilver: Fast predictive image registration– A deep learning approach, NeuroImage, 158 (2017), 378-396. doi: 10.1016/j.neuroimage.2017.07.008.
[33]	J. Yoo, S. Sabir, D. Heo, K. H. Kim, A. Wahab, Y. Choi, et al., Deep learning diffuse optical tomography, IEEE Transactions on Medical Imaging, 39 (2020), 877–887. doi: 10.1109/TMI.2019.2936522.