American Institute of Mathematical Sciences

Advanced
doi: 10.3934/ipi.2020051

LANTERN: Learn analysis transform network for dynamic magnetic resonance imaging

Shanshan Wang 1, , Yanxia Chen 1,2, , Taohui Xiao 1, , Lei Zhang 1, , Xin Liu 1, and  Hairong Zheng 1,,

1. 

Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, Guangdong, China, 1068 Xueyuan Avenue, Shenzhen University Town, Shenzhen, China
2. 

University of Chinese Academy of Sciences, Beijing, China, 19 Yuquan Road, Shijingshan District, Beijing, China

* Corresponding author: Hairong Zheng

S. Wang and Y. Chen contributed equally to this work

Received  December 2019 Revised  May 2020 Published  August 2020

This paper proposes to learn analysis transform network for dynamic magnetic resonance imaging (LANTERN). Integrating the strength of CS-MRI and deep learning, the proposed framework is highlighted in three components: (ⅰ) The spatial and temporal domains are sparsely constrained by adaptively trained convolutional filters; (ⅱ) We introduce an end-to-end framework to learn the parameters in LANTERN to solve the difficulty of parameter selection in traditional methods; (ⅲ) Compared to existing deep learning reconstruction methods, our experimental results show that our paper has encouraging capability in exploiting the spatial and temporal redundancy of dynamic MR images. We performed quantitative and qualitative analysis of cardiac reconstructions at different acceleration factors ($ 2 \times $-$ 11 \times $) with different undersampling patterns. In comparison with two state-of-the-art methods, experimental results show that our method achieved encouraging performances.

Keywords: Dynamic MR imaging, Deep learning, CS-MRI.
Mathematics Subject Classification: Primary: 58F15, 58F17; Secondary: 53C35.
Citation: Shanshan Wang, Yanxia Chen, Taohui Xiao, Lei Zhang, Xin Liu, Hairong Zheng. LANTERN: Learn analysis transform network for dynamic magnetic resonance imaging. Inverse Problems & Imaging, doi: 10.3934/ipi.2020051
References:
[1]

H. K. Aggarwal, M. P. Mani and M. Jacob, Modl: Model-based deep learning architecture for inverse problems, IEEE Transactions on Medical Imaging, 38 (2018), 394-405.  doi: 10.1109/TMI.2018.2865356.  Google Scholar

[2]

K. T. Block, M. Uecker and J. Frahm, Undersampled radial MRI with multiple coils. Iterative image reconstruction using a total variation constraint, Magnetic Resonance in Medicine, 57 (2007), 1086-1098.  doi: 10.1002/mrm.21236.  Google Scholar

[3]

J. Caballero, A. N. Price, D. Rueckert and J. V Hajnal, Dictionary learning and time sparsity for dynamic MR data reconstruction, IEEE Transactions on Medical Imaging, 33 (2014), 979-994.  doi: 10.1109/TMI.2014.2301271.  Google Scholar

[4]

L. Chaari, J. C. Pesquet, A. Benazza-Benyahia and P. Ciuciu, A wavelet-based regularized reconstruction algorithm for SENSE parallel MRI with applications to neuroimaging, Med. Image Anal, 15 (2011), 185-201.  doi: 10.1016/j.media.2010.08.001.  Google Scholar

[5]

T. Eo, Y. Jun, T. Kim, J. Jang, H. Lee and D. Hwang, KIKI-Net: Cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images, Magnetic Resonance in Medicine, 80 (2018), 2188-2201.  doi: 10.1002/mrm.27201.  Google Scholar

[6]

K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock and F. Knoll, Learning a variational network for reconstruction of accelerated MRI data, Magnetic Resonance in Medicine, 79 (2018), 3055-3071.  doi: 10.1002/mrm.26977.  Google Scholar

[7]

Y. Han, J. Yoo, H. H. Kim, H. J. Shin, K. Sung and J. C. Ye, Deep learning with domain adaptation for accelerated projection–reconstruction MR, Magnetic Resonance in Medicine, 80 (2018), 1189-1205.   Google Scholar

[8]

H. Jung, K. Sung, K. S. Nayak, E. Y. Kim and J. C. Ye, K-T FOCUSS: A general compressed sensing framework for high resolution dynamic MRI, Magnetic Resonance in Medicine. An Off. J. Int. Soc. Magn. Reson. Med, 61 (2009), 103-116.   Google Scholar

[9]

H. Jung, J. Yoo and J. C. Ye, Generalized kt BLAST and kt SENSE using FOCUSS, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE, (2007), 145–148. doi: 10.1109/ISBI.2007.356809.  Google Scholar

[10]

H. Jung and J. C. Ye, Motion estimated and compensated compressed sensing dynamic magnetic resonance imaging: What we can learn from video compression techniques, Int. J. Imaging Syst. Technol, 20 (2010), 81-98.  doi: 10.1002/ima.20231.  Google Scholar

[11]

W. A. Kaiser and E. Zeitler, MR imaging of the breast: Fast imaging sequences with and without Gd-DTPA. Preliminary observations, Radiology, 170 (1989), 681-686.  doi: 10.1148/radiology.170.3.2916021.  Google Scholar

[12]

F. Knoll, T. Murrell, A. Sriram, N. Yakubova, J. Zbontar, M. Rabbat, A. Defazio, M. J. Muckley, D. K. Sodickson, C. L. Zitnick and M. P. Recht, Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge, arXiv preprint, arXiv: 2001.02518, 2020. Google Scholar

[13]

D. Liang, J. Cheng and Z. Ke Z, Deep magnetic resonance image reconstruction: Inverse problems meet neural networks, IEEE Signal Processing Magazine, 37 (2020), 141-151.  doi: 10.1109/MSP.2019.2950557.  Google Scholar

[14]

D. Liang, E. V. R. DiBella, R. R. Chen and L. Ying, K-t ISD: Dynamic cardiac MR imaging using compressed sensing with iterative support detection, Magnetic Resonance in Medicine, 68 (2012), 41-53.  doi: 10.1002/mrm.23197.  Google Scholar

[15]

S. G. Lingala, Y. Hu, E. Dibella and M. Jacob, Accelerated dynamic MRI exploiting sparsity and low-rank structure: K-t SLR, IEEE Transactions on Medical Imaging, 30 (2011), 1042-1054.  doi: 10.1109/TMI.2010.2100850.  Google Scholar

[16]

Q. Liu, Q. Yang, H. Cheng, S. Wang, M. Zhang and D. Liang, highly undersampled magnetic resonance imaging reconstruction using autoencoder priors, Magnetic Resonance in Medicine, 83 (2020), 322-336.  doi: 10.1002/mrm.27921.  Google Scholar

[17]

F. Liu, D. Li, X. Jin, W. Qiu, Q. Xia and B. Sun, Dynamic cardiac MRI reconstruction using motion aligned locally low rank tensor (MALLRT), Magnetic Resonance in Medicine, 66 (2020), 104-115.  doi: 10.1016/j.mri.2019.07.002.  Google Scholar

[18]

Y. Liu, Q. Liu, M. Zhang, Q. Yang, S. Wang and D. Liang, IFR-net: Iterative feature refinement net-work for compressed sensing MRI, IEEE Transactions on Computational Imaging, 6 (2019), 434-446.  doi: 10.1109/TCI.2019.2956877.  Google Scholar

[19]

M. Lustig, J. M. Santos, D. L. Donoho and J. M. Pauly, KT sparse: high frame-rate dynamic magnetic resonance imaging exploiting spatio-temporal sparsity, U.S. Patent, 7 (2009), 183. Google Scholar

[20]

A. Majumdar, Improved dynamic MRI reconstruction by exploiting sparsity and rank-deficiency, Magn. Reson. Imaging, 31 (2013), 789-795.  doi: 10.1016/j.mri.2012.10.026.  Google Scholar

[21]

A. Majumdar, R. K. Ward and T. Aboulnasr, Non-convex algorithm for sparse and low-rank recovery: Application to dynamic MRI reconstruction, Magn. Reson. Imaging, 31 (2013), 448-455.  doi: 10.1016/j.mri.2012.08.011.  Google Scholar

[22]

A. Majumdar, Improving synthesis and analysis prior blind compressed sensing with low-rank constraints for dynamic MRI reconstruction, Magn. Reson. Imaging, 33 (2015), 174-179.  doi: 10.1016/j.mri.2014.08.031.  Google Scholar

[23]

S. Osher, M. Burger, D. Goldfarb, J. Xu and W. Yin, An iterative regularization method for total variation-based image restoration, Multiscale Model. Simul, 4 (2005), 460-489.  doi: 10.1137/040605412.  Google Scholar

[24]

R. Otazo, E. Cands and D. K. Sodickson, Low-rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components, Magnetic Resonance in Medicine, 73 (2015), 1125-1136.  doi: 10.1002/mrm.25240.  Google Scholar

[25]

T. M. Quan, T. Nguyen-Duc and W.-K. Jeong, Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss, IEEE Transactions on Medical Imaging, 37 (2018), 1488-1497.  doi: 10.1109/TMI.2018.2820120.  Google Scholar

[26]

C. Qin, J. Schlemper, J. Caballero, A. N. Price, J. V Hajnal and D. Rueckert, Convolutional recurrent neural networks for dynamic MR image reconstruction, IEEE Transactions on Medical Imaging, 38 (2019), 280-290.  doi: 10.1109/TMI.2018.2863670.  Google Scholar

[27]

M. Rizkinia and M. Okuda, Evaluation of primal-dual splitting algorithm for MRI reconstruction using spatio-temporal structure Tensor and L1-2 norm, Makara Journal of Technology, 23 (2020), 126-130.  doi: 10.7454/mst.v23i3.3892.  Google Scholar

[28]

D. k. Sodickson and W. J. Manning, Simultaneous acquisition of spatial harmonics (SMASH): Fast imaging with radiofrequency coil arrays, Magnetic Resonance in Medicine, 38 (1997), 591-603.  doi: 10.1002/mrm.1910380414.  Google Scholar

[29]

J. Schlemper, J. Caballero, J. V Hajnal, A. N. Price and D. Ruecker, A deep cascade of convolutional neural networks for dynamic MR image reconstruction, IEEE Transactions on Medical Imaging, 37 (2018), 491-503.  doi: 10.1109/TMI.2017.2760978.  Google Scholar

[30]

J. Sun, H. Li and Z. Xu, Deep ADMM-Net for compressive sensing MRI, Advances in Neural Information Processing Systems, (2016), 10–18. http://papers.nips.cc/paper/6406-deep-admm-net-for-compressive-sensing-mri. Google Scholar

[31]

L. Sun, Z. Fan, Y. Huang, X. Ding and J. Paisley, Compressed sensing MRI using a recursive dilated network, Thirty-Second AAAI Conference on Artificial Intelligence, (2018). http://www.columbia.edu/ jwp2128/Papers/SunFanetal2018.pdf Google Scholar

[32]

J. Tsao, P. Boesiger and K. P. Pruessmann, k-t BLAST and k-t SENSE: Dynamic MRI With High Frame Rate Exploiting Spatiotemporal Correlations, Magnetic Resonance in Medicine, 50 (2003), 1031-1042.  doi: 10.1002/mrm.10611.  Google Scholar

[33]

S. Wang, Y. Xia, Q. Liu, P. Dong and D. Feng, Fenchel duality based dictionary learning for restoration of noisy images, IEEE Transactions on Image Processing, 22 (2013), 5214-5225.  doi: 10.1109/TIP.2013.2282900.  Google Scholar

[34]

Y. Wang, Y. Zhou and L. Ying, Undersampled dynamic magnetic resonance imaging using patch-based spatiotemporal dictionaries, 2013 IEEE 10th International Symposium on Biomedical Imaging, (2013), 294–297. doi: 10.1109/ISBI.2013.6556470.  Google Scholar

[35]

S. Wang, Z. Ke, H. Cheng, S. Jia, Y. Leslie, H. Zheng and D. Liang, Dimension: Dynamic mr imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training, NMR in Biomedicine, (2019), e4131. doi: 10.1002/nbm.4131.  Google Scholar

[36]

S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang, D. Feng and D. Liang, Accelerating magnetic resonance imaging via deep learning, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), (2016). doi: 10.1109/ISBI.2016.7493320.  Google Scholar

[37]

S. Wang, H. Cheng, L. Ying, T. Xiao, Z. Ke, H. Zheng and D. Liang, DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution, Magnetic Resonance in Medicine, 68 (2020), 136-147.  doi: 10.1016/j.mri.2020.02.002.  Google Scholar

[38]

Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, Image quality assessment: From error visibility to structural similarity, IEEE Transactions on Image Processing, 13 (2004), 600–612. https://ece.uwaterloo.ca/ z70wang/publications/ssim.pdf doi: 10.1109/TIP.2003.819861.  Google Scholar

[39]

J. Yao, Z. Xu, X. Huang and J Huang, Accelerated dynamic MRI reconstruction with total variation and nuclear norm regularization, International Conference on Medical Image Computing and Computer-Assisted Intervention, (2015), 635–642. doi: 10.1007/978-3-319-24571-3_76.  Google Scholar

[40]

G. Yang, S. Yu and H. Dong, DAGAN: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction, IEEE Transactions on Medical Imaging, 37 (2018), 1310-1321.  doi: 10.1109/TMI.2017.2785879.  Google Scholar

[41]

Y. Yang, J. Sun, H. Li and Z Xu, ADMM-CSNet: A deep learning approach for image compressive sensing, IEEE Transactions on Pattern Analysis and Machine Intelligence, 42 (2018), 521-538.   Google Scholar

[42]

B. Zhao, J. P. Haldar, A. G. Christodoulou and Z.-P. Liang, Image reconstruction from highly undersampled (k, t)-space data with joint partial separability and sparsity constraints, IEEE Transactions on Medical Imaging, 31 (2012), 1809-1820.  doi: 10.1109/TMI.2012.2203921.  Google Scholar

[43]

B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Rosen and M. S. Rosen, Image reconstruction by domain-transform manifold learning, Nature, 555 (2018), 487-492.  doi: 10.1038/nature25988.  Google Scholar

show all references

References:
[1]

H. K. Aggarwal, M. P. Mani and M. Jacob, Modl: Model-based deep learning architecture for inverse problems, IEEE Transactions on Medical Imaging, 38 (2018), 394-405.  doi: 10.1109/TMI.2018.2865356.  Google Scholar

[2]

K. T. Block, M. Uecker and J. Frahm, Undersampled radial MRI with multiple coils. Iterative image reconstruction using a total variation constraint, Magnetic Resonance in Medicine, 57 (2007), 1086-1098.  doi: 10.1002/mrm.21236.  Google Scholar

[3]

J. Caballero, A. N. Price, D. Rueckert and J. V Hajnal, Dictionary learning and time sparsity for dynamic MR data reconstruction, IEEE Transactions on Medical Imaging, 33 (2014), 979-994.  doi: 10.1109/TMI.2014.2301271.  Google Scholar

[4]

L. Chaari, J. C. Pesquet, A. Benazza-Benyahia and P. Ciuciu, A wavelet-based regularized reconstruction algorithm for SENSE parallel MRI with applications to neuroimaging, Med. Image Anal, 15 (2011), 185-201.  doi: 10.1016/j.media.2010.08.001.  Google Scholar

[5]

T. Eo, Y. Jun, T. Kim, J. Jang, H. Lee and D. Hwang, KIKI-Net: Cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images, Magnetic Resonance in Medicine, 80 (2018), 2188-2201.  doi: 10.1002/mrm.27201.  Google Scholar

[6]

K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock and F. Knoll, Learning a variational network for reconstruction of accelerated MRI data, Magnetic Resonance in Medicine, 79 (2018), 3055-3071.  doi: 10.1002/mrm.26977.  Google Scholar

[7]

Y. Han, J. Yoo, H. H. Kim, H. J. Shin, K. Sung and J. C. Ye, Deep learning with domain adaptation for accelerated projection–reconstruction MR, Magnetic Resonance in Medicine, 80 (2018), 1189-1205.   Google Scholar

[8]

H. Jung, K. Sung, K. S. Nayak, E. Y. Kim and J. C. Ye, K-T FOCUSS: A general compressed sensing framework for high resolution dynamic MRI, Magnetic Resonance in Medicine. An Off. J. Int. Soc. Magn. Reson. Med, 61 (2009), 103-116.   Google Scholar

[9]

H. Jung, J. Yoo and J. C. Ye, Generalized kt BLAST and kt SENSE using FOCUSS, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE, (2007), 145–148. doi: 10.1109/ISBI.2007.356809.  Google Scholar

[10]

H. Jung and J. C. Ye, Motion estimated and compensated compressed sensing dynamic magnetic resonance imaging: What we can learn from video compression techniques, Int. J. Imaging Syst. Technol, 20 (2010), 81-98.  doi: 10.1002/ima.20231.  Google Scholar

[11]

W. A. Kaiser and E. Zeitler, MR imaging of the breast: Fast imaging sequences with and without Gd-DTPA. Preliminary observations, Radiology, 170 (1989), 681-686.  doi: 10.1148/radiology.170.3.2916021.  Google Scholar

[12]

F. Knoll, T. Murrell, A. Sriram, N. Yakubova, J. Zbontar, M. Rabbat, A. Defazio, M. J. Muckley, D. K. Sodickson, C. L. Zitnick and M. P. Recht, Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge, arXiv preprint, arXiv: 2001.02518, 2020. Google Scholar

[13]

D. Liang, J. Cheng and Z. Ke Z, Deep magnetic resonance image reconstruction: Inverse problems meet neural networks, IEEE Signal Processing Magazine, 37 (2020), 141-151.  doi: 10.1109/MSP.2019.2950557.  Google Scholar

[14]

D. Liang, E. V. R. DiBella, R. R. Chen and L. Ying, K-t ISD: Dynamic cardiac MR imaging using compressed sensing with iterative support detection, Magnetic Resonance in Medicine, 68 (2012), 41-53.  doi: 10.1002/mrm.23197.  Google Scholar

[15]

S. G. Lingala, Y. Hu, E. Dibella and M. Jacob, Accelerated dynamic MRI exploiting sparsity and low-rank structure: K-t SLR, IEEE Transactions on Medical Imaging, 30 (2011), 1042-1054.  doi: 10.1109/TMI.2010.2100850.  Google Scholar

[16]

Q. Liu, Q. Yang, H. Cheng, S. Wang, M. Zhang and D. Liang, highly undersampled magnetic resonance imaging reconstruction using autoencoder priors, Magnetic Resonance in Medicine, 83 (2020), 322-336.  doi: 10.1002/mrm.27921.  Google Scholar

[17]

F. Liu, D. Li, X. Jin, W. Qiu, Q. Xia and B. Sun, Dynamic cardiac MRI reconstruction using motion aligned locally low rank tensor (MALLRT), Magnetic Resonance in Medicine, 66 (2020), 104-115.  doi: 10.1016/j.mri.2019.07.002.  Google Scholar

[18]

Y. Liu, Q. Liu, M. Zhang, Q. Yang, S. Wang and D. Liang, IFR-net: Iterative feature refinement net-work for compressed sensing MRI, IEEE Transactions on Computational Imaging, 6 (2019), 434-446.  doi: 10.1109/TCI.2019.2956877.  Google Scholar

[19]

M. Lustig, J. M. Santos, D. L. Donoho and J. M. Pauly, KT sparse: high frame-rate dynamic magnetic resonance imaging exploiting spatio-temporal sparsity, U.S. Patent, 7 (2009), 183. Google Scholar

[20]

A. Majumdar, Improved dynamic MRI reconstruction by exploiting sparsity and rank-deficiency, Magn. Reson. Imaging, 31 (2013), 789-795.  doi: 10.1016/j.mri.2012.10.026.  Google Scholar

[21]

A. Majumdar, R. K. Ward and T. Aboulnasr, Non-convex algorithm for sparse and low-rank recovery: Application to dynamic MRI reconstruction, Magn. Reson. Imaging, 31 (2013), 448-455.  doi: 10.1016/j.mri.2012.08.011.  Google Scholar

[22]

A. Majumdar, Improving synthesis and analysis prior blind compressed sensing with low-rank constraints for dynamic MRI reconstruction, Magn. Reson. Imaging, 33 (2015), 174-179.  doi: 10.1016/j.mri.2014.08.031.  Google Scholar

[23]

S. Osher, M. Burger, D. Goldfarb, J. Xu and W. Yin, An iterative regularization method for total variation-based image restoration, Multiscale Model. Simul, 4 (2005), 460-489.  doi: 10.1137/040605412.  Google Scholar

[24]

R. Otazo, E. Cands and D. K. Sodickson, Low-rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components, Magnetic Resonance in Medicine, 73 (2015), 1125-1136.  doi: 10.1002/mrm.25240.  Google Scholar

[25]

T. M. Quan, T. Nguyen-Duc and W.-K. Jeong, Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss, IEEE Transactions on Medical Imaging, 37 (2018), 1488-1497.  doi: 10.1109/TMI.2018.2820120.  Google Scholar

[26]

C. Qin, J. Schlemper, J. Caballero, A. N. Price, J. V Hajnal and D. Rueckert, Convolutional recurrent neural networks for dynamic MR image reconstruction, IEEE Transactions on Medical Imaging, 38 (2019), 280-290.  doi: 10.1109/TMI.2018.2863670.  Google Scholar

[27]

M. Rizkinia and M. Okuda, Evaluation of primal-dual splitting algorithm for MRI reconstruction using spatio-temporal structure Tensor and L1-2 norm, Makara Journal of Technology, 23 (2020), 126-130.  doi: 10.7454/mst.v23i3.3892.  Google Scholar

[28]

D. k. Sodickson and W. J. Manning, Simultaneous acquisition of spatial harmonics (SMASH): Fast imaging with radiofrequency coil arrays, Magnetic Resonance in Medicine, 38 (1997), 591-603.  doi: 10.1002/mrm.1910380414.  Google Scholar

[29]

J. Schlemper, J. Caballero, J. V Hajnal, A. N. Price and D. Ruecker, A deep cascade of convolutional neural networks for dynamic MR image reconstruction, IEEE Transactions on Medical Imaging, 37 (2018), 491-503.  doi: 10.1109/TMI.2017.2760978.  Google Scholar

[30]

J. Sun, H. Li and Z. Xu, Deep ADMM-Net for compressive sensing MRI, Advances in Neural Information Processing Systems, (2016), 10–18. http://papers.nips.cc/paper/6406-deep-admm-net-for-compressive-sensing-mri. Google Scholar

[31]

L. Sun, Z. Fan, Y. Huang, X. Ding and J. Paisley, Compressed sensing MRI using a recursive dilated network, Thirty-Second AAAI Conference on Artificial Intelligence, (2018). http://www.columbia.edu/ jwp2128/Papers/SunFanetal2018.pdf Google Scholar

[32]

J. Tsao, P. Boesiger and K. P. Pruessmann, k-t BLAST and k-t SENSE: Dynamic MRI With High Frame Rate Exploiting Spatiotemporal Correlations, Magnetic Resonance in Medicine, 50 (2003), 1031-1042.  doi: 10.1002/mrm.10611.  Google Scholar

[33]

S. Wang, Y. Xia, Q. Liu, P. Dong and D. Feng, Fenchel duality based dictionary learning for restoration of noisy images, IEEE Transactions on Image Processing, 22 (2013), 5214-5225.  doi: 10.1109/TIP.2013.2282900.  Google Scholar

[34]

Y. Wang, Y. Zhou and L. Ying, Undersampled dynamic magnetic resonance imaging using patch-based spatiotemporal dictionaries, 2013 IEEE 10th International Symposium on Biomedical Imaging, (2013), 294–297. doi: 10.1109/ISBI.2013.6556470.  Google Scholar

[35]

S. Wang, Z. Ke, H. Cheng, S. Jia, Y. Leslie, H. Zheng and D. Liang, Dimension: Dynamic mr imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training, NMR in Biomedicine, (2019), e4131. doi: 10.1002/nbm.4131.  Google Scholar

[36]

S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang, D. Feng and D. Liang, Accelerating magnetic resonance imaging via deep learning, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), (2016). doi: 10.1109/ISBI.2016.7493320.  Google Scholar

[37]

S. Wang, H. Cheng, L. Ying, T. Xiao, Z. Ke, H. Zheng and D. Liang, DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution, Magnetic Resonance in Medicine, 68 (2020), 136-147.  doi: 10.1016/j.mri.2020.02.002.  Google Scholar

[38]

Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, Image quality assessment: From error visibility to structural similarity, IEEE Transactions on Image Processing, 13 (2004), 600–612. https://ece.uwaterloo.ca/ z70wang/publications/ssim.pdf doi: 10.1109/TIP.2003.819861.  Google Scholar

[39]

J. Yao, Z. Xu, X. Huang and J Huang, Accelerated dynamic MRI reconstruction with total variation and nuclear norm regularization, International Conference on Medical Image Computing and Computer-Assisted Intervention, (2015), 635–642. doi: 10.1007/978-3-319-24571-3_76.  Google Scholar

[40]

G. Yang, S. Yu and H. Dong, DAGAN: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction, IEEE Transactions on Medical Imaging, 37 (2018), 1310-1321.  doi: 10.1109/TMI.2017.2785879.  Google Scholar

[41]

Y. Yang, J. Sun, H. Li and Z Xu, ADMM-CSNet: A deep learning approach for image compressive sensing, IEEE Transactions on Pattern Analysis and Machine Intelligence, 42 (2018), 521-538.   Google Scholar

[42]

B. Zhao, J. P. Haldar, A. G. Christodoulou and Z.-P. Liang, Image reconstruction from highly undersampled (k, t)-space data with joint partial separability and sparsity constraints, IEEE Transactions on Medical Imaging, 31 (2012), 1809-1820.  doi: 10.1109/TMI.2012.2203921.  Google Scholar

[43]

B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Rosen and M. S. Rosen, Image reconstruction by domain-transform manifold learning, Nature, 555 (2018), 487-492.  doi: 10.1038/nature25988.  Google Scholar

Figure 1.  The proposed LANTERN network architecture for dMRI reconstruction. In (A) and (B), the blue arrow indicates forward process. The pink arrow indicates the process of back-propagation to update network parameters, where $ i $ represents the $ i - th $ iteration and $ N_i $ represents a total of $ N_i $ iterations. $ k $ expresses that the priori loop for $ k $ times and $ (i, k) $ means that in the $ i-th $ iteration, the a priori loops $ k $ times
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Visual results comparison for the sensitivity to the training data size. From left to right, the reconstruction results (top line) with neural networks trained from different amounts of data based on the proposed method with 1D random sampling pattern at an acceleration factor of 4. PSNR values are given in the middle and the reconstruction error maps are presented at the bottom
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  The comparison of the three initialization modes of Random Gaussian, TV, DCT and LANTERN based on the proposed method with 1D Random sampling at an acceleration factor of 4. PSNR value are given under the results
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  The comparison of k-t SLR, D5C5 and the proposed method with 1D Random sampling at an acceleration factor of 4. PSNR value is given under the results
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  The comparison of k-t SLR, D5C5 and the proposed method with 1D Random sampling at an acceleration factor of 5. PSNR value is given under the results
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  The comparison of various methods between average quantification index of the 50 test data and acceleration factor based on 1D Random sampling
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 8.  The comparison of various methods between average quantification index of the 50 test data and acceleration factor based on Radial sampling
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 9.  The training and validation loss curves of the proposed model
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7.  The comparison of k-t SLR, D5C5 and the proposed method with 2D Radial sampling at an acceleration factor of 11. PSNR value is given under the results
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  Experimental masks and acceleration factors
$\bf{1D\ Random}$ $\bf{2D\ Radial}$
2X 3X 4X 5X 7X 9X 11X 2X 3X 4X 5X 7X 9X 11X 15X
Table Options

Download as excel

$\bf{1D\ Random}$ $\bf{2D\ Radial}$
2X 3X 4X 5X 7X 9X 11X 2X 3X 4X 5X 7X 9X 11X 15X
Table 2.  Quantitative results comparison for the sensitivity to the training data size. The average quantitative indicator values of the results reconstructed for the 50 test data with the network trained from different different amount of data with 1D Random sampling pattern at an accelerated factor of 4
1D Random4x NMSE PSNR/dB SSIM HFEN
data50 0.0413 40.8047 0.8943 0.8333
data60 0.0397 41.1515 0.9 0.7939
data80 0.0388 41.3589 0.9034 0.7729
data100 0.0385 41.4391 0.9043 0.7633
data120 0.0386 41.4402 0.9035 0.7685
Table Options

Download as excel

1D Random4x NMSE PSNR/dB SSIM HFEN
data50 0.0413 40.8047 0.8943 0.8333
data60 0.0397 41.1515 0.9 0.7939
data80 0.0388 41.3589 0.9034 0.7729
data100 0.0385 41.4391 0.9043 0.7633
data120 0.0386 41.4402 0.9035 0.7685
Table 3.  Quantitative results comparison for the sensitivity to the initialization. The average quantitative indicator values of the results reconstructed for the 50 test data with the network trained with different initialization with 1D Random sampling pattern at an accelerated factor of 4
1D Random4x Gaussian TV DCT LANTERN
PSNR HFEN PSNR HFEN PSNR HFEN PSNR HFEN
AVE 39.8089 0.9459 40.5884 0.8514 40.9971 0.8064 41.4391 0.7633
Table Options

Download as excel

1D Random4x Gaussian TV DCT LANTERN
PSNR HFEN PSNR HFEN PSNR HFEN PSNR HFEN
AVE 39.8089 0.9459 40.5884 0.8514 40.9971 0.8064 41.4391 0.7633
Table 4.  Average reconstruction quantitative metrics with standard deviation of the 50 test data based on various methods with 1D Random sampling at a different accelerated factor
Methods Random 7X Random 11X
PSNR SSIM HFEN PSNR SSIM HFEN
Zero-filling 29.14$ \pm $2.2 0.57$ \pm $0.04 2.73$ \pm $0.65 27.58$ \pm $2.08 0.51$ \pm $0.04 3.12$ \pm $0.74
Kt-SLR 33.50$ \pm $2.70 0.77$ \pm $0.03 1.83$ \pm $0.53 32.44$ \pm $2.61 0.73$ \pm $0.03 2.01$ \pm $0.63
D5C5 36.76$ \pm $2.00 0.78$ \pm $0.03 1.40$ \pm $0.33 35.22$ \pm $2.00 0.73$ \pm $0.03 1.82$ \pm $0.50
Proposed 37.48$ \pm $2.45 0.82$ \pm $0.02 1.31$ \pm $0.36 35.40$ \pm $2.60 0.77$ \pm $0.03 1.67$ \pm $0.53
Table Options

Download as excel

Methods Random 7X Random 11X
PSNR SSIM HFEN PSNR SSIM HFEN
Zero-filling 29.14$ \pm $2.2 0.57$ \pm $0.04 2.73$ \pm $0.65 27.58$ \pm $2.08 0.51$ \pm $0.04 3.12$ \pm $0.74
Kt-SLR 33.50$ \pm $2.70 0.77$ \pm $0.03 1.83$ \pm $0.53 32.44$ \pm $2.61 0.73$ \pm $0.03 2.01$ \pm $0.63
D5C5 36.76$ \pm $2.00 0.78$ \pm $0.03 1.40$ \pm $0.33 35.22$ \pm $2.00 0.73$ \pm $0.03 1.82$ \pm $0.50
Proposed 37.48$ \pm $2.45 0.82$ \pm $0.02 1.31$ \pm $0.36 35.40$ \pm $2.60 0.77$ \pm $0.03 1.67$ \pm $0.53
Table 5.  Average reconstruction quantitative metrics with standard deviation of the 50 test data based on various methods with radial sampling at a different accelerated factor
Methods Radial 11X Radial 15X
PSNR/dB SSIM HFEN PSNR/dB SSIM HFEN
Zero-filling 22.269$ \pm $1.37 0.345$ \pm $0.06 5.198$ \pm $0.72 20.153$ \pm $1.27 0.275$ \pm $0.05 5.986$ \pm $0.67
Kt-SLR 31.961$ \pm $2.34 0.718$ \pm $0.03 2.179$ \pm $0.51 31.518$ \pm $2.36 0.707$ \pm $0.04 2.229$ \pm $0.54
D5C5 34.954$ \pm $2.08 0.701$ \pm $0.03 1.735$ \pm $0.42 34.248$ \pm $2.04 0.677$ \pm $0.03 1.907$ \pm $0.45
Proposed 38.874$ \pm $2.28 0.831$ \pm $0.03 1.019$ \pm $0.26 38.115$ \pm $2.23 0.808$ \pm $0.03 1.164$ \pm $0.30
Table Options

Download as excel

Methods Radial 11X Radial 15X
PSNR/dB SSIM HFEN PSNR/dB SSIM HFEN
Zero-filling 22.269$ \pm $1.37 0.345$ \pm $0.06 5.198$ \pm $0.72 20.153$ \pm $1.27 0.275$ \pm $0.05 5.986$ \pm $0.67
Kt-SLR 31.961$ \pm $2.34 0.718$ \pm $0.03 2.179$ \pm $0.51 31.518$ \pm $2.36 0.707$ \pm $0.04 2.229$ \pm $0.54
D5C5 34.954$ \pm $2.08 0.701$ \pm $0.03 1.735$ \pm $0.42 34.248$ \pm $2.04 0.677$ \pm $0.03 1.907$ \pm $0.45
Proposed 38.874$ \pm $2.28 0.831$ \pm $0.03 1.019$ \pm $0.26 38.115$ \pm $2.23 0.808$ \pm $0.03 1.164$ \pm $0.30
[1]

Xianchao Xiu, Lingchen Kong. Rank-one and sparse matrix decomposition for dynamic MRI. Numerical Algebra, Control & Optimization, 2015, 5 (2) : 127-134. doi: 10.3934/naco.2015.5.127
[2]

Martin Benning, Elena Celledoni, Matthias J. Ehrhardt, Brynjulf Owren, Carola-Bibiane Schönlieb. Deep learning as optimal control problems: Models and numerical methods. Journal of Computational Dynamics, 2019, 6 (2) : 171-198. doi: 10.3934/jcd.2019009
[3]

Nicholas Geneva, Nicholas Zabaras. Multi-fidelity generative deep learning turbulent flows. Foundations of Data Science, 2020, 2 (4) : 391-428. doi: 10.3934/fods.2020019
[4]

Miria Feng, Wenying Feng. Evaluation of parallel and sequential deep learning models for music subgenre classification. Mathematical Foundations of Computing, 2021, 4 (2) : 131-143. doi: 10.3934/mfc.2021008
[5]

Tim McGraw, Baba Vemuri, Evren Özarslan, Yunmei Chen, Thomas Mareci. Variational denoising of diffusion weighted MRI. Inverse Problems & Imaging, 2009, 3 (4) : 625-648. doi: 10.3934/ipi.2009.3.625
[6]

H. N. Mhaskar, T. Poggio. Function approximation by deep networks. Communications on Pure & Applied Analysis, 2020, 19 (8) : 4085-4095. doi: 10.3934/cpaa.2020181
[7]

Jerry L. Bona, Henrik Kalisch. Models for internal waves in deep water. Discrete & Continuous Dynamical Systems, 2000, 6 (1) : 1-20. doi: 10.3934/dcds.2000.6.1
[8]

Robert D. Sidman, Marie Erie, Henry Chu. A method, with applications, for analyzing co-registered EEG and MRI data. Conference Publications, 2001, 2001 (Special) : 349-356. doi: 10.3934/proc.2001.2001.349
[9]

Ryan Compton, Stanley Osher, Louis-S. Bouchard. Hybrid regularization for MRI reconstruction with static field inhomogeneity correction. Inverse Problems & Imaging, 2013, 7 (4) : 1215-1233. doi: 10.3934/ipi.2013.7.1215
[10]

Yuyuan Ouyang, Yunmei Chen, Ying Wu. Total variation and wavelet regularization of orientation distribution functions in diffusion MRI. Inverse Problems & Imaging, 2013, 7 (2) : 565-583. doi: 10.3934/ipi.2013.7.565
[11]

Daniela Calvetti, Erkki Somersalo. Microlocal sequential regularization in imaging. Inverse Problems & Imaging, 2007, 1 (1) : 1-11. doi: 10.3934/ipi.2007.1.1
[12]

Guillaume Bal, Olivier Pinaud, Lenya Ryzhik. On the stability of some imaging functionals. Inverse Problems & Imaging, 2016, 10 (3) : 585-616. doi: 10.3934/ipi.2016013
[13]

Laurent Bourgeois, Jean-François Fritsch, Arnaud Recoquillay. Imaging junctions of waveguides. Inverse Problems & Imaging, 2021, 15 (2) : 285-314. doi: 10.3934/ipi.2020065
[14]

Anca-Voichita Matioc. On particle trajectories in linear deep-water waves. Communications on Pure & Applied Analysis, 2012, 11 (4) : 1537-1547. doi: 10.3934/cpaa.2012.11.1537
[15]

Jiequn Han, Jihao Long. Convergence of the deep BSDE method for coupled FBSDEs. Probability, Uncertainty and Quantitative Risk, 2020, 5 (0) : 5-. doi: 10.1186/s41546-020-00047-w
[16]

Lars Grüne. Computing Lyapunov functions using deep neural networks. Journal of Computational Dynamics, 2021, 8 (2) : 131-152. doi: 10.3934/jcd.2021006
[17]

Yunmei Chen, Xiaojing Ye, Feng Huang. A novel method and fast algorithm for MR image reconstruction with significantly under-sampled data. Inverse Problems & Imaging, 2010, 4 (2) : 223-240. doi: 10.3934/ipi.2010.4.223
[18]

Piotr Gwiazda, Filip Z. Klawe, Agnieszka Świerczewska-Gwiazda. Thermo-visco-elasticity for the Mróz model in the framework of thermodynamically complete systems. Discrete & Continuous Dynamical Systems - S, 2014, 7 (5) : 981-991. doi: 10.3934/dcdss.2014.7.981
[19]

Hyeuknam Kwon, Yoon Mo Jung, Jaeseok Park, Jin Keun Seo. A new computer-aided method for detecting brain metastases on contrast-enhanced MR images. Inverse Problems & Imaging, 2014, 8 (2) : 491-505. doi: 10.3934/ipi.2014.8.491
[20]

Alan Beggs. Learning in monotone bayesian games. Journal of Dynamics & Games, 2015, 2 (2) : 117-140. doi: 10.3934/jdg.2015.2.117

2019 Impact Factor: 1.373

Tools

Article outline

Figures and Tables

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences