• Previous Article
    Reconstructing a potential perturbation of the biharmonic operator on transversally anisotropic manifolds
  • IPI Home
  • This Issue
  • Next Article
    Source identification problems for abstract semilinear nonlocal differential equations
doi: 10.3934/ipi.2022037
Online First

Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Online First articles via the “Online First” tab for the selected journal.

Data-consistent neural networks for solving nonlinear inverse problems

1. 

Department of Applied Mathematics, University of Twente, Netherlands

2. 

Department of Mathematics, University of Innsbruck, Austria

3. 

Department of Mathematics, University of Manchester, United Kingdom

*Corresponding author: Johannes Schwab

Received  May 2021 Revised  May 2022 Early access July 2022

Data assisted reconstruction algorithms, incorporating trained neural networks, are a novel paradigm for solving inverse problems. One approach is to first apply a classical reconstruction method and then apply a neural network to improve its solution. Empirical evidence shows that plain two-step methods provide high-quality reconstructions, but they lack a convergence analysis as known for classical regularization methods. In this paper we formalize the use of such two-step approaches in the context of classical regularization theory. We propose data-consistent neural networks that can be combined with classical regularization methods. This yields a data-driven regularization method for which we provide a convergence analysis with respect to noise. Numerical simulations show that compared to standard two-step deep learning methods, our approach provides better stability with respect to out of distribution examples in the test set, while performing similarly on test data drawn from the distribution of the training set. Our method provides a stable solution approach to inverse problems that beneficially combines the known nonlinear forward model with available information on the desired solution manifold in training data.

Citation: Yoeri E. Boink, Markus Haltmeier, Sean Holman, Johannes Schwab. Data-consistent neural networks for solving nonlinear inverse problems. Inverse Problems and Imaging, doi: 10.3934/ipi.2022037
References:
[1]

A. Adler, V. Emiya, M. G. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, A constrained matching pursuit approach to audio declipping, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (2011), 329–332. doi: 10.1109/ICASSP.2011.5946407.

[2]

J. Adler and O. Öktem, Solving ill-posed inverse problems using iterative deep neural networks, Inverse Probl., 33 (2017), 124007.  doi: 10.1088/1361-6420/aa9581.

[3]

S. AntholzerM. Haltmeier and J. Schwab, Deep learning for photoacoustic tomography from sparse data, Inverse Problems in Science and Engineering, 27 (2019), 987-1005.  doi: 10.1080/17415977.2018.1518444.

[4]

S. Armato IIIG. McLennanL. BidautM. McNitt-GrayC. MeyerA. ReevesB. ZhaoD. AberleC. Henschke and E. Hoffman, The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans, Medical Physics, 38 (2011), 915-931. 

[5]

S. ArridgeP. MaassO. Öktem and C. B. Schönlieb, Solving inverse problems using data-driven models, Acta Numerica, 28 (2019), 1-174.  doi: 10.1017/S0962492919000059.

[6]

A. B. Bakushinsky and M. Yu. Kokurin, Iterative Methods for Approximate Solution of Inverse Problems, Springer, 2004

[7]

H. H. Bauschke and J. M. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Review, 38 (1996), 367-426.  doi: 10.1137/S0036144593251710.

[8]

Y. Boink and C. Brune, Learned SVD: Solving inverse problems via hybrid autoencoding, arXiv: 1912.10840, (2020).

[9]

Y. BoinkS. Manohar and C. Brune, A partially learned algorithm for joint photoacoustic reconstruction and segmentation, IEEE Transactions on Medical Imaging, 39 (2020), 129-139.  doi: 10.1109/TMI.2019.2922026.

[10]

T. A. BubbaG. KutyniokM. LassasM. MärzW. SamekS. Siltanen and V. Srinivasan, Learning the invisible: A hybrid deep learning-shearlet framework for limited angle computed tomography, Inverse Problems, 35 (2019), 064002.  doi: 10.1088/1361-6420/ab10ca.

[11]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers Group, 1996.

[12]

D. Gilton, G. Ongie and R. Willett, Neumann networks for inverse problems in imaging, IEEE Trans. Comput. Imaging, 6 (2020), 328–343. arXiv: 1901.03707, (2019). doi: 10.1109/TCI.2019.2948732.

[13]

H. GoukE. FrankB. Pfahringer and M. J. Cree, Regularisation of neural networks by enforcing lipschitz continuity, Machine Learning, 110 (2021), 393-416.  doi: 10.1007/s10994-020-05929-w.

[14]

I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin and A. Courville, Improved training of wasserstein gans, arXiv Preprint, arXiv: 1704.00028, (2017).

[15]

S. J. Hamilton and A. Hauptmann, Deep D-bar: Real time electrical impedance tomography imaging with deep neural networks, IEEE Trans. Med. Imag., 37 (2018), 2367-2377.  doi: 10.1109/TMI.2018.2828303.

[16]

P. C. Hansen, Discrete Inverse Problems: Insight and Algorithms, SIAM, 2010. doi: 10.1137/1.9780898718836.

[17]

K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770–778. doi: 10.1109/CVPR.2016.90.

[18]

S. Honig and M. Werman, Image declipping with deep networks, 2018 25th IEEE International Conference on Image Processing (ICIP), (2018), 3923–3927. doi: 10.1109/ICIP.2018.8451780.

[19]

M. V. de Hoop, M. Lassas and C. A. Wong, Deep learning architectures for nonlinear operator functions and nonlinear inverse problems, Math. Stat. Learn., 4 (2021), 1–86. arXiv: 1912.11090, (2019). doi: 10.4171/msl/28.

[20]

K. H. JinM. T. McCannE. Froustey and M. Unser, Deep convolutional neural network for inverse problems in imaging, IEEE Trans. Image Process., 26 (2017), 4509-4522.  doi: 10.1109/TIP.2017.2713099.

[21]

B. Kaltenbacher, A. Neubauer and O. Scherzer, Iterative Regularization Methods for Nonlinear Ill-Posed Problems, , Walter de Gruyter GmbH & Co. KG, Berlin, 2008. doi: 10.1515/9783110208276.

[22]

E. Kobler, T. Klatzer, K. Hammernik and T. Pock, Variational networks: Connecting variational methods and deep learning, German Conference on Pattern Recognition, (2017), 281–293. doi: 10.1007/978-3-319-66709-6.

[23]

A. Kofler, M. Haltmeier, C. Kolbitsch, M. Kachelrieß and M. Dewey, A U-Nets cascade for sparse view computed tomography, International Workshop on Machine Learning for Medical Image Reconstruction, (2018), 91–99. doi: 10.1007/978-3-030-00129-2_11.

[24]

D. Lee, J. Yoo and J. C. Ye, Deep residual learning for compressed sensing MRI, IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), 15–18. doi: 10.1109/ISBI.2017.7950457.

[25]

R. M. Lewitt, Multidimensional digital image representations using generalized Kaiser-Bessel window functions, JOSA A, 7 (1990), 1834-1846.  doi: 10.1364/JOSAA.7.001834.

[26]

J. Leuschner, M. Schmidt, D. Baguer and P. Maaß, The LoDoPaB-CT dataset: A benchmark dataset for low-dose CT reconstruction methods, arXiv: 1910.01113, (2019).

[27]

H. LiJ. SchwabS. Antholzer and M. Haltmeier, NETT: Solving inverse problems with deep neural networks, Inverse Problems, Online First, 36 (2020), 065005.  doi: 10.1088/1361-6420/ab6d57.

[28]

S. Lunz, O. Öktem and C. Schönlieb, Adversarial regularizers in inverse problems, Advances in Neural Information Processing Systems, (2018), 8507–8516.

[29]

M. Mardani, E. Gong, J. Cheng, S. Vasanawala, G. Zaharchuk, M. Alley, N. Thakur, W. Han, J. Pauly, et al., Deep generative adversarial networks for compressed sensing automates MRI, arXiv: 1706.00051, (2017)

[30]

V. A. Morozov, Methods for Solving Incorrectly Posed Problems, Springer Verlag, 1984. doi: 10.1007/978-1-4612-5280-1.

[31]

S. Mukherjee, S. Dittmer, Z. Shumaylov, S. Lunz, O. Öktem and C. Schönlieb, Learned convex regularizers for inverse problems, arXiv Preprint, arXiv: 2008.02839, (2020).

[32]

J. Rick Chang, C. Li, B. Poczos, B. Vijaya and A. Sankaranarayanan, One network to solve them all-solving linear inverse problems using deep projection models, Proceedings of The IEEE International Conference on Computer Vision, (2017), 5888–5897. doi: 10.1109/ICCV.2017.627.

[33]

Y. RivensonZ. GöröcsH. GünaydinY. ZhangH. Wang and A. Ozcan, Deep learning microscopy, Optica, 4 (2017), 1437-1443.  doi: 10.1364/OPTICA.4.001437.

[34]

O. Ronneberger, P. Fischer and T. Brox, U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-Assisted Intervention, (2015), 234–241. doi: 10.1007/978-3-319-24574-4_28.

[35]

O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier and F. Lenzen, Variational Methods in Imaging, Springer, 2009.

[36]

J. SchwabS. Antholzer and M. Haltmeier, Deep null space learning for inverse problems: Convergence analysis and rates, Inverse Problems, 35 (2019), 025008.  doi: 10.1088/1361-6420/aaf14a.

[37]

J. SchwabS. Antholzer and M. Haltmeier, Big in Japan: Regularizing networks for solving inverse problems, Journal of Mathematical Imaging and Vision, 62 (2020), 445-455.  doi: 10.1007/s10851-019-00911-1.

[38]

J. Sun, H. Li, Z. Xu, et al., Deep ADMM-Net for compressive sensing MRI, Advances in Neural Information Processing Systems, (2016), 10–18.

[39]

A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems, John Wiley & Sons, 1977.

[40]

X. ZhangJ. Wang and L. Xing, Metal artifact reduction in x-ray computed tomography (CT) by constrained optimization, Medical Physics, 38 (2011), 701-711.  doi: 10.1118/1.3533711.

[41]

B. ZhuJ. Z. LiuS. F. CauleyB. R. Rosen and M. S. Rosen, Image reconstruction by domain-transform manifold learning, Nature, 555 (2018), 487-492.  doi: 10.1038/nature25988.

show all references

References:
[1]

A. Adler, V. Emiya, M. G. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, A constrained matching pursuit approach to audio declipping, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (2011), 329–332. doi: 10.1109/ICASSP.2011.5946407.

[2]

J. Adler and O. Öktem, Solving ill-posed inverse problems using iterative deep neural networks, Inverse Probl., 33 (2017), 124007.  doi: 10.1088/1361-6420/aa9581.

[3]

S. AntholzerM. Haltmeier and J. Schwab, Deep learning for photoacoustic tomography from sparse data, Inverse Problems in Science and Engineering, 27 (2019), 987-1005.  doi: 10.1080/17415977.2018.1518444.

[4]

S. Armato IIIG. McLennanL. BidautM. McNitt-GrayC. MeyerA. ReevesB. ZhaoD. AberleC. Henschke and E. Hoffman, The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans, Medical Physics, 38 (2011), 915-931. 

[5]

S. ArridgeP. MaassO. Öktem and C. B. Schönlieb, Solving inverse problems using data-driven models, Acta Numerica, 28 (2019), 1-174.  doi: 10.1017/S0962492919000059.

[6]

A. B. Bakushinsky and M. Yu. Kokurin, Iterative Methods for Approximate Solution of Inverse Problems, Springer, 2004

[7]

H. H. Bauschke and J. M. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Review, 38 (1996), 367-426.  doi: 10.1137/S0036144593251710.

[8]

Y. Boink and C. Brune, Learned SVD: Solving inverse problems via hybrid autoencoding, arXiv: 1912.10840, (2020).

[9]

Y. BoinkS. Manohar and C. Brune, A partially learned algorithm for joint photoacoustic reconstruction and segmentation, IEEE Transactions on Medical Imaging, 39 (2020), 129-139.  doi: 10.1109/TMI.2019.2922026.

[10]

T. A. BubbaG. KutyniokM. LassasM. MärzW. SamekS. Siltanen and V. Srinivasan, Learning the invisible: A hybrid deep learning-shearlet framework for limited angle computed tomography, Inverse Problems, 35 (2019), 064002.  doi: 10.1088/1361-6420/ab10ca.

[11]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers Group, 1996.

[12]

D. Gilton, G. Ongie and R. Willett, Neumann networks for inverse problems in imaging, IEEE Trans. Comput. Imaging, 6 (2020), 328–343. arXiv: 1901.03707, (2019). doi: 10.1109/TCI.2019.2948732.

[13]

H. GoukE. FrankB. Pfahringer and M. J. Cree, Regularisation of neural networks by enforcing lipschitz continuity, Machine Learning, 110 (2021), 393-416.  doi: 10.1007/s10994-020-05929-w.

[14]

I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin and A. Courville, Improved training of wasserstein gans, arXiv Preprint, arXiv: 1704.00028, (2017).

[15]

S. J. Hamilton and A. Hauptmann, Deep D-bar: Real time electrical impedance tomography imaging with deep neural networks, IEEE Trans. Med. Imag., 37 (2018), 2367-2377.  doi: 10.1109/TMI.2018.2828303.

[16]

P. C. Hansen, Discrete Inverse Problems: Insight and Algorithms, SIAM, 2010. doi: 10.1137/1.9780898718836.

[17]

K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770–778. doi: 10.1109/CVPR.2016.90.

[18]

S. Honig and M. Werman, Image declipping with deep networks, 2018 25th IEEE International Conference on Image Processing (ICIP), (2018), 3923–3927. doi: 10.1109/ICIP.2018.8451780.

[19]

M. V. de Hoop, M. Lassas and C. A. Wong, Deep learning architectures for nonlinear operator functions and nonlinear inverse problems, Math. Stat. Learn., 4 (2021), 1–86. arXiv: 1912.11090, (2019). doi: 10.4171/msl/28.

[20]

K. H. JinM. T. McCannE. Froustey and M. Unser, Deep convolutional neural network for inverse problems in imaging, IEEE Trans. Image Process., 26 (2017), 4509-4522.  doi: 10.1109/TIP.2017.2713099.

[21]

B. Kaltenbacher, A. Neubauer and O. Scherzer, Iterative Regularization Methods for Nonlinear Ill-Posed Problems, , Walter de Gruyter GmbH & Co. KG, Berlin, 2008. doi: 10.1515/9783110208276.

[22]

E. Kobler, T. Klatzer, K. Hammernik and T. Pock, Variational networks: Connecting variational methods and deep learning, German Conference on Pattern Recognition, (2017), 281–293. doi: 10.1007/978-3-319-66709-6.

[23]

A. Kofler, M. Haltmeier, C. Kolbitsch, M. Kachelrieß and M. Dewey, A U-Nets cascade for sparse view computed tomography, International Workshop on Machine Learning for Medical Image Reconstruction, (2018), 91–99. doi: 10.1007/978-3-030-00129-2_11.

[24]

D. Lee, J. Yoo and J. C. Ye, Deep residual learning for compressed sensing MRI, IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), 15–18. doi: 10.1109/ISBI.2017.7950457.

[25]

R. M. Lewitt, Multidimensional digital image representations using generalized Kaiser-Bessel window functions, JOSA A, 7 (1990), 1834-1846.  doi: 10.1364/JOSAA.7.001834.

[26]

J. Leuschner, M. Schmidt, D. Baguer and P. Maaß, The LoDoPaB-CT dataset: A benchmark dataset for low-dose CT reconstruction methods, arXiv: 1910.01113, (2019).

[27]

H. LiJ. SchwabS. Antholzer and M. Haltmeier, NETT: Solving inverse problems with deep neural networks, Inverse Problems, Online First, 36 (2020), 065005.  doi: 10.1088/1361-6420/ab6d57.

[28]

S. Lunz, O. Öktem and C. Schönlieb, Adversarial regularizers in inverse problems, Advances in Neural Information Processing Systems, (2018), 8507–8516.

[29]

M. Mardani, E. Gong, J. Cheng, S. Vasanawala, G. Zaharchuk, M. Alley, N. Thakur, W. Han, J. Pauly, et al., Deep generative adversarial networks for compressed sensing automates MRI, arXiv: 1706.00051, (2017)

[30]

V. A. Morozov, Methods for Solving Incorrectly Posed Problems, Springer Verlag, 1984. doi: 10.1007/978-1-4612-5280-1.

[31]

S. Mukherjee, S. Dittmer, Z. Shumaylov, S. Lunz, O. Öktem and C. Schönlieb, Learned convex regularizers for inverse problems, arXiv Preprint, arXiv: 2008.02839, (2020).

[32]

J. Rick Chang, C. Li, B. Poczos, B. Vijaya and A. Sankaranarayanan, One network to solve them all-solving linear inverse problems using deep projection models, Proceedings of The IEEE International Conference on Computer Vision, (2017), 5888–5897. doi: 10.1109/ICCV.2017.627.

[33]

Y. RivensonZ. GöröcsH. GünaydinY. ZhangH. Wang and A. Ozcan, Deep learning microscopy, Optica, 4 (2017), 1437-1443.  doi: 10.1364/OPTICA.4.001437.

[34]

O. Ronneberger, P. Fischer and T. Brox, U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-Assisted Intervention, (2015), 234–241. doi: 10.1007/978-3-319-24574-4_28.

[35]

O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier and F. Lenzen, Variational Methods in Imaging, Springer, 2009.

[36]

J. SchwabS. Antholzer and M. Haltmeier, Deep null space learning for inverse problems: Convergence analysis and rates, Inverse Problems, 35 (2019), 025008.  doi: 10.1088/1361-6420/aaf14a.

[37]

J. SchwabS. Antholzer and M. Haltmeier, Big in Japan: Regularizing networks for solving inverse problems, Journal of Mathematical Imaging and Vision, 62 (2020), 445-455.  doi: 10.1007/s10851-019-00911-1.

[38]

J. Sun, H. Li, Z. Xu, et al., Deep ADMM-Net for compressive sensing MRI, Advances in Neural Information Processing Systems, (2016), 10–18.

[39]

A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems, John Wiley & Sons, 1977.

[40]

X. ZhangJ. Wang and L. Xing, Metal artifact reduction in x-ray computed tomography (CT) by constrained optimization, Medical Physics, 38 (2011), 701-711.  doi: 10.1118/1.3533711.

[41]

B. ZhuJ. Z. LiuS. F. CauleyB. R. Rosen and M. S. Rosen, Image reconstruction by domain-transform manifold learning, Nature, 555 (2018), 487-492.  doi: 10.1038/nature25988.

Figure 1.  On the left, a standard post-processing network. The red box illustrates that the output in general does not reproduce the data under the forward operation $ \mathcal{F} $. On the right a data-consistent network architecture. The green box illustrates that it does reproduce the data under the forward operation $ \mathcal{F} $
Figure 2.  The figure illustrates the solution set for given data $ y $ and the output of the network $ \mathbf{U} $ acting on the solution after applying a right inverse of $ \mathcal{F} $. The generalized projection operator approximated by some iterative scheme or closed form operator is shown in green and blue respectively
Figure 3.  Two-dimensional visualization of the data-consistent network for the 'projection on convex set' problem explained in Section 4.1. The blue region indicates the convex set $ C $, the green region (that extends infinitely to the right) indicates the affine normal cone to $ C $ at $ \mathcal{P}_C(z) $. The output of the data-consistent network is indicated by $ \Phi_0(z) $; it can be seen that it is obtained by taking the input $ z $, applying a Lipschitz continuous neural network $ \mathbf{U}(z) $ and projecting it to the normal cone $ \mathcal N_C $ at $ \mathcal{P}_C(z) $. This ensures that $ \mathcal{P}_C( \Phi_0(z)) = \mathcal{P}_C(z) $, as required by the definition of a data-consistent network
Figure 7.  Reconstructions of a typical sample from the modified test set. Top: reconstructed sinograms with all 8 angles in different colors. Bottom: reconstructed images (grayscale from 0 to 1)
Figure 4.  Reconstructions of a sample from the regular test set. In the bottom the horizontal central slice is shown. Both U-Net and data-consistent network provide an almost perfect reconstruction
Figure 5.  Reconstructions of a sample from the modified test set. In the bottom the horizontal central slice is shown. Data-consistency makes sure that intensity is only changed above the saturation level
Figure 6.  Reconstructions of a typical sample from the regular test set. Top: reconstructed sinograms with all 8 angles in different colors. Bottom: reconstructed images (grayscale from 0 to 1)
Figure 11.  Sample for which the data-consistent PSNR value is relatively high compared to the U-Net PSNR values (grayscale from 0 to 1)
Figure 12.  Sample for which the data-consistent PSNR value is approximately the same as the U-Net PSNR values (grayscale from 0 to 1)
Figure 13.  Sample for which the data-invariant PSNR value is relatively low compared to the U-Net PSNR values (grayscale from 0 to 1)
Figure 14.  Sample for which the data-invariant PSNR value is relatively high compared to the U-Net PSNR values (grayscale from 0 to 1)
Figure 15.  Sample for which the data-invariant PSNR value is approximately the same as the U-Net PSNR values (grayscale from 0 to 1)
Figure 16.  Sample for which the data-invariant PSNR value is relatively low compared to the U-Net PSNR values (grayscale from 0 to 1)
Table 1.  U-Net parameter details for all simulation experiments
Exp. 1 $ ( \mathbf{U}/ {{\mathit{\boldsymbol{\Phi}}}}_0) $: Exp. 2 $ ( \mathbf{U}_1/ {{\mathit{\boldsymbol{\Phi}}}}_0^{(1)}) $: Exp. 2 $ ( \mathbf{U}_2/ {{\mathit{\boldsymbol{\Phi}}}}_0^{(2)}) $:
image domain image domain sinogram domain
$ \# $training samples 1024 35584 35584
$ \# $validation samples 256 3522 3522
$ \# $test samples 1024 3553 3553
depth 4 4 4
width 2 2 2
$ \# $channels in top layer 8 16 16
convolution size $ 3\times3 $ $ 3\times3 $ $ 3\times3 $
nonlinearity ReLU ReLU ReLU
start learning rate $ 10^{-3} $ $ 10^{-3} $ $ 10^{-3} $
final learning rate $ 10^{-4} $ $ 2\cdot10^{-4} $ $ 2\cdot10^{-4} $
batch size 64 32 32
$ \# $epochs 1000 25 25
Exp. 1 $ ( \mathbf{U}/ {{\mathit{\boldsymbol{\Phi}}}}_0) $: Exp. 2 $ ( \mathbf{U}_1/ {{\mathit{\boldsymbol{\Phi}}}}_0^{(1)}) $: Exp. 2 $ ( \mathbf{U}_2/ {{\mathit{\boldsymbol{\Phi}}}}_0^{(2)}) $:
image domain image domain sinogram domain
$ \# $training samples 1024 35584 35584
$ \# $validation samples 256 3522 3522
$ \# $test samples 1024 3553 3553
depth 4 4 4
width 2 2 2
$ \# $channels in top layer 8 16 16
convolution size $ 3\times3 $ $ 3\times3 $ $ 3\times3 $
nonlinearity ReLU ReLU ReLU
start learning rate $ 10^{-3} $ $ 10^{-3} $ $ 10^{-3} $
final learning rate $ 10^{-4} $ $ 2\cdot10^{-4} $ $ 2\cdot10^{-4} $
batch size 64 32 32
$ \# $epochs 1000 25 25
Table 2.  Comparison of PSNR and SSIM for all reconstruction methods
PSNR SSIM
Pseudo-inverse U-Net Data-consistent Pseudo-inverse U-Net Data-consistent
Regular set $ 24.2 \pm2.2 $ $ 60.6 \pm2.1 $ $ 66.7 \pm1.6 $ $ 0.56 \pm 0.08 $ $ 1.00 \pm0.00 $ $ 1.00 \pm0.00 $
Modified set $ 48.0 \pm7.8 $ $ 36.9 \pm2.9 $ $ 48.0 \pm4.4 $ $ 0.99 \pm 0.01 $ $ 0.92 \pm0.03 $ $ 0.97 \pm0.01 $
PSNR SSIM
Pseudo-inverse U-Net Data-consistent Pseudo-inverse U-Net Data-consistent
Regular set $ 24.2 \pm2.2 $ $ 60.6 \pm2.1 $ $ 66.7 \pm1.6 $ $ 0.56 \pm 0.08 $ $ 1.00 \pm0.00 $ $ 1.00 \pm0.00 $
Modified set $ 48.0 \pm7.8 $ $ 36.9 \pm2.9 $ $ 48.0 \pm4.4 $ $ 0.99 \pm 0.01 $ $ 0.92 \pm0.03 $ $ 0.97 \pm0.01 $
Table 3.  Comparison of PSNR, SSIM and data-fidelity $ \Vert{ \mathbf{F}(\tilde{ x})- \mathbf{F}( x)}\Vert_{\ell^2} $ for all reconstruction methods, where $ \tilde{ x} $ is the reconstruction and $ x $ is the ground truth. Note that the data-fidelity of the pseudo-inverse in theory should be zero since it is a right inverse of the forward operator. The non-zero values result from numerical instabilities in the computation of this operator
PSNR
Pseudo-inverse One U-Net Two U-Nets Data-consistent
Regular set $ 23.1 \pm2.3 $ $ 30.5 \pm1.5 $ $ 31.0 \pm1.5 $ $ 30.1 \pm1.9 $
Modified set $ 29.1 \pm1.6 $ $ 27.5 \pm1.7 $ $ 28.3 \pm1.4 $ $ 29.9 \pm1.2 $
SSIM
Pseudo-inverse One U-Net Two U-Nets Data-consistent
Regular set $ 0.50 \pm0.07 $ $ 0.82 \pm0.04 $ $ 0.83 \pm0.04 $ $ 0.74 \pm0.07 $
Modified set $ 0.71 \pm0.07 $ $ 0.74 \pm0.05 $ $ 0.73 \pm0.05 $ $ 0.75 \pm0.05 $
Data-fidelity
Pseudo-inverse One U-Net Two U-Nets Data-consistent
Regular set $ 6.1 \pm3.3 $ $ \quad 4.8 \pm1.5 $ $ 3.9 \pm1.1 $ $ 0.9 \pm0.4 $
Modified set $ 0.4 \pm0.2 $ $ 11.9 \pm5.4 $ $ 8.5 \pm2.8 $ $ 0.6 \pm0.2 $
PSNR
Pseudo-inverse One U-Net Two U-Nets Data-consistent
Regular set $ 23.1 \pm2.3 $ $ 30.5 \pm1.5 $ $ 31.0 \pm1.5 $ $ 30.1 \pm1.9 $
Modified set $ 29.1 \pm1.6 $ $ 27.5 \pm1.7 $ $ 28.3 \pm1.4 $ $ 29.9 \pm1.2 $
SSIM
Pseudo-inverse One U-Net Two U-Nets Data-consistent
Regular set $ 0.50 \pm0.07 $ $ 0.82 \pm0.04 $ $ 0.83 \pm0.04 $ $ 0.74 \pm0.07 $
Modified set $ 0.71 \pm0.07 $ $ 0.74 \pm0.05 $ $ 0.73 \pm0.05 $ $ 0.75 \pm0.05 $
Data-fidelity
Pseudo-inverse One U-Net Two U-Nets Data-consistent
Regular set $ 6.1 \pm3.3 $ $ \quad 4.8 \pm1.5 $ $ 3.9 \pm1.1 $ $ 0.9 \pm0.4 $
Modified set $ 0.4 \pm0.2 $ $ 11.9 \pm5.4 $ $ 8.5 \pm2.8 $ $ 0.6 \pm0.2 $
[1]

Hyeontae Jo, Hwijae Son, Hyung Ju Hwang, Eun Heui Kim. Deep neural network approach to forward-inverse problems. Networks and Heterogeneous Media, 2020, 15 (2) : 247-259. doi: 10.3934/nhm.2020011

[2]

Lars Grüne. Computing Lyapunov functions using deep neural networks. Journal of Computational Dynamics, 2021, 8 (2) : 131-152. doi: 10.3934/jcd.2021006

[3]

Daniel Gerth, Andreas Hofinger, Ronny Ramlau. On the lifting of deterministic convergence rates for inverse problems with stochastic noise. Inverse Problems and Imaging, 2017, 11 (4) : 663-687. doi: 10.3934/ipi.2017031

[4]

Abhishake Rastogi. Tikhonov regularization with oversmoothing penalty for nonlinear statistical inverse problems. Communications on Pure and Applied Analysis, 2020, 19 (8) : 4111-4126. doi: 10.3934/cpaa.2020183

[5]

Stefan Kindermann, Antonio Leitão. Convergence rates for Kaczmarz-type regularization methods. Inverse Problems and Imaging, 2014, 8 (1) : 149-172. doi: 10.3934/ipi.2014.8.149

[6]

Bruno Sixou, Cyril Mory. Kullback-Leibler residual and regularization for inverse problems with noisy data and noisy operator. Inverse Problems and Imaging, 2019, 13 (5) : 1113-1137. doi: 10.3934/ipi.2019050

[7]

Martin Benning, Elena Celledoni, Matthias J. Ehrhardt, Brynjulf Owren, Carola-Bibiane Schönlieb. Deep learning as optimal control problems: Models and numerical methods. Journal of Computational Dynamics, 2019, 6 (2) : 171-198. doi: 10.3934/jcd.2019009

[8]

Thorsten Hohage, Mihaela Pricop. Nonlinear Tikhonov regularization in Hilbert scales for inverse boundary value problems with random noise. Inverse Problems and Imaging, 2008, 2 (2) : 271-290. doi: 10.3934/ipi.2008.2.271

[9]

Zhiyan Ding, Qin Li, Jianfeng Lu. Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds. Foundations of Data Science, 2021, 3 (3) : 371-411. doi: 10.3934/fods.2020018

[10]

Gabriel Peyré, Sébastien Bougleux, Laurent Cohen. Non-local regularization of inverse problems. Inverse Problems and Imaging, 2011, 5 (2) : 511-530. doi: 10.3934/ipi.2011.5.511

[11]

Philipp Hungerländer, Barbara Kaltenbacher, Franz Rendl. Regularization of inverse problems via box constrained minimization. Inverse Problems and Imaging, 2020, 14 (3) : 437-461. doi: 10.3934/ipi.2020021

[12]

De-han Chen, Daijun jiang. Convergence rates of Tikhonov regularization for recovering growth rates in a Lotka-Volterra competition model with diffusion. Inverse Problems and Imaging, 2021, 15 (5) : 951-974. doi: 10.3934/ipi.2021023

[13]

Jie Zhao. Convergence rates for elliptic reiterated homogenization problems. Communications on Pure and Applied Analysis, 2013, 12 (6) : 2787-2795. doi: 10.3934/cpaa.2013.12.2787

[14]

Leong-Kwan Li, Sally Shao. Convergence analysis of the weighted state space search algorithm for recurrent neural networks. Numerical Algebra, Control and Optimization, 2014, 4 (3) : 193-207. doi: 10.3934/naco.2014.4.193

[15]

Xiaosheng Li, Gunther Uhlmann. Inverse problems with partial data in a slab. Inverse Problems and Imaging, 2010, 4 (3) : 449-462. doi: 10.3934/ipi.2010.4.449

[16]

Ruhua Wang, Senjian An, Wanquan Liu, Ling Li. Fixed-point algorithms for inverse of residual rectifier neural networks. Mathematical Foundations of Computing, 2021, 4 (1) : 31-44. doi: 10.3934/mfc.2020024

[17]

Michele La Rocca, Cira Perna. Designing neural networks for modeling biological data: A statistical perspective. Mathematical Biosciences & Engineering, 2014, 11 (2) : 331-342. doi: 10.3934/mbe.2014.11.331

[18]

H. N. Mhaskar, T. Poggio. Function approximation by deep networks. Communications on Pure and Applied Analysis, 2020, 19 (8) : 4085-4095. doi: 10.3934/cpaa.2020181

[19]

Mina Jiang, Changjiang Zhu. Convergence rates to nonlinear diffusion waves for $p$-system with nonlinear damping on quadrant. Discrete and Continuous Dynamical Systems, 2009, 23 (3) : 887-918. doi: 10.3934/dcds.2009.23.887

[20]

Seonho Park, Maciej Rysz, Kaitlin L. Fair, Panos M. Pardalos. Synthetic-Aperture Radar image based positioning in GPS-denied environments using Deep Cosine Similarity Neural Networks. Inverse Problems and Imaging, 2021, 15 (4) : 763-785. doi: 10.3934/ipi.2021013

2021 Impact Factor: 1.483

Metrics

  • PDF downloads (29)
  • HTML views (19)
  • Cited by (0)

[Back to Top]