Complex extension of optical flow and its practical evaluation for undersampled dynamic MRI

Matthias J. Ehrhardt; Marco Mauritz

doi:10.3934/ammc.2025005

Article Contents

March 2025, Volume 3, 117-133. Doi: 10.3934/ammc.2025005

This volume Previous Article Solution of an inverse problem to estimate airway resistance in mechanically ventilated patients from 3-D electrical impedance tomography data Next Article

Complex extension of optical flow and its practical evaluation for undersampled dynamic MRI

Matthias J. Ehrhardt^1, and
Marco Mauritz^2, ,

1.
University of Bath, United Kingdom
2.
University of Münster, Germany

^*Corresponding author: Marco Mauritz
^*Corresponding author: Marco Mauritz

Received: December 2024

Early access: March 2025

Published: March 2025

Abstract / Introduction Full Text(HTML) Figure(7) / Table(1) Related Papers Cited by

Abstract

Reconstructing high-quality images from undersampled dynamic MRI data is a challenging task and important for the success of this imaging modality. To remedy the naturally occurring artifacts due to measurement undersampling, one can incorporate a motion model into the reconstruction so that information can propagate across time frames. Current models for MRI imaging are using the optical flow equation. However, they are based on real-valued images. Here, we generalise the optical flow equation to complex-valued images and demonstrate, based on two real cardiac MRI datasets, that the new model is capable of improving image quality.

Keywords:

Mathematics Subject Classification: Primary: 92C55; Secondary: 65K10.

Citation:

FullText(HTML)

Figure 1. Comparison between real-valued and complex-valued reconstructions in MRI. Left: Complex-valued reconstruction. Right: Real-valued reconstruction. Top row: Magnitude representation. The bottom row shows the same images but as a complex-valued image: The hue represents the normalised phase and the brightness represents the magnitude. The red boxes point to some missing structures in the real-valued image

Download: Full-size image PowerPoint slide

Figure 2. Each row shows a reconstruction for a different data set. The first column shows a dynamic ground truth (obtained as a reconstruction using (4) and fully-sampled data). The second column shows the same for a frame-wise ground truth (obtained from reconstructing each time frame separately ($\texttt{FW} $ model) using fully-sampled data). The last column shows an overlay of the reconstructions with the dynamic masks. Within each column, the left side shows one time slice of the reconstructions, the right side shows time-space images along the red lines

Download: Full-size image PowerPoint slide

Figure 3. Results for the simulated data. The top row shows the evolution (magnitude) of the simulated ground truth images for the time points $ t_0 $, $ t_2 $, $ t_4 $, $ t_6 $ (out of $ 8 $ time points). The second row shows the same images but as a complex-valued image: The hue represents the normalised phase and the brightness represents the magnitude. The other rows depict the difference between the magnitudes of the reconstructions and the magnitude of the simulated images on the dynamic mask. Rows 3-5: $\texttt{FW} $, $ \texttt{DT}$ and $ \texttt{OF}$

Download: Full-size image PowerPoint slide

Figure 4. Reconstructed velocities of one time frame for the simulated data using our proposed complex-valued optical flow method. First row: Ground truth velocities in $ x $ and $ y $ direction. Second row: Reconstructed velocities in $ x $ and $ y $ direction

Download: Full-size image PowerPoint slide

Figure 5. Reconstructions for subject 1. Left to right columns: Ground truth image, $\texttt{FW} $, $\texttt{DT} $, $ \texttt{OF}$ and $\texttt{Cheat-OF} $. First row shows magnitude of reconstructions. Second row: Difference of reconstruction to ground truth on the mask. Third row: Magnified area and corresponding metrics on magnification

Download: Full-size image PowerPoint slide

Figure 6. Reconstructions for subject 2. Left to right columns: Ground truth image, $\texttt{FW} $, $\texttt{DT} $, $ \texttt{OF}$ and $\texttt{Cheat-OF} $. First and second row show magnitude of reconstructions with respective metrics. Third and fourth row show the phase of the reconstructions. Second and last row are the indicated magnified areas

Download: Full-size image PowerPoint slide

Figure 7. Comparison between ground truth velocities (top row) and reconstructed velocities (bottom row) of subject 1 for a single time frame

Download: Full-size image PowerPoint slide

Table 1. Summary of the results. The columns show the metrics for the respective data sets (with dynamic ground truth GT(dyn) and frame-wise ground truth GT(FW)). The displayed values are $ (\text{average})\pm(\text{standard deviation}) $ taken over the considered time points. The metric for each time point is computed on a dynamic mask (see section 4.2, fig. 2)

		simulation	subject 1 GT(dyn)	subject 2 GT(dyn)	subject 1 GT(FW)	subject 2 GT(FW)

PSNR	$\texttt{FW} $	$ 34.1\pm1.2 $	$ 27.3\pm0.7 $	$ 25.6\pm0.6 $	$ 27.7\pm 0.8 $	$ 26.4\pm 0.9 $
	$\texttt{DT} $	$ 40.1\pm0.9 $	$ 29.6\pm0.3 $	$ 26.2\pm0.3 $	$ 29.8\pm 0.4 $	$ 27.5\pm 0.5 $
	$ \texttt{OF}$	$ 44.1\pm0.6 $	$ 30.2\pm0.5 $	$ 27.6\pm0.4 $	$ 30.2\pm 0.5 $	$ 28.0\pm 0.5 $
	$\texttt{Cheat-OF} $	$ 44.2\pm0.3 $	$ 34.4\pm0.7 $	$ 32.0\pm0.7 $	$ 33.1\pm 0.4 $	$ 28.7\pm 1.0 $

SSIM	$ \texttt{FW}$	$ 0.943\pm0.010 $	$ 0.828\pm0.014 $	$ 0.812\pm0.015 $	$ 0.847\pm 0.014 $	$ 0.852\pm 0.018 $
	$\texttt{DT} $	$ 0.978\pm0.004 $	$ 0.869\pm0.009 $	$ 0.819\pm0.012 $	$ 0.880\pm 0.009 $	$ 0.869\pm0.011 $
	$\texttt{OF} $	$ 0.991\pm0.001 $	$ 0.892\pm0.010 $	$ 0.873\pm0.012 $	$ 0.888\pm 0.011 $	$ 0.880\pm 0.011 $
	$\texttt{Cheat-OF} $	$ 0.992\pm0.001 $	$ 0.959\pm0.009 $	$ 0.942\pm0.009 $	$ 0.934\pm 0.004 $	$ 0.872\pm 0.018 $

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, The Clarendon Press, Oxford University Press, New York, 2000.
[2]	A. I. Aviles-Rivero, N. Debroux, G. Williams, M. J. Graves and C.-B. Schönlieb, Compressed sensing plus motion (cs + m): A new perspective for improving undersampled mr image reconstruction, Med Image Anal., 68 (2021), 101933. doi: 10.1016/j.media.2020.101933.
[3]	E. C. Bassett, E. G. Kholmovski, B. D. Wilson, E. V. R. DiBella, D. J. Dosdall, R. Ranjan, C. J. McGann and D. Kim, Evaluation of highly accelerated real-time cardiac cine mri in tachycardia, NMR in Biomedicine, 27 (2014), 175-182. doi: 10.1002/nbm.3049.
[4]	A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM Journal on Imaging Sciences, 2 (2009), 183-202. doi: 10.1137/080716542.
[5]	J.-D. Benamou and Y. Brenier, A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem, Numer. Math., 84 (2000), 375-393. doi: 10.1007/s002110050002.
[6]	J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne and Q. Zhang, JAX: Composable transformations of Python+NumPy programs, 2018.
[7]	M. Burger, H. Dirks, L. Frerking, A. Hauptmann, T. Helin and S. Siltanen, A variational reconstruction method for undersampled dynamic x-ray tomography based on physical motion models, Inverse Problems, 33 (2017), 124008, 24pp. doi: 10.1088/1361-6420/aa99cf.
[8]	M. Burger, H. Dirks and C.-B. Schönlieb, A variational model for joint motion estimation and image reconstruction, SIAM Journal on Imaging Sciences, 11 (2018), 94-128. doi: 10.1137/16M1084183.
[9]	C. Chen, B. Gris and O. Öktem, A new variational model for joint image reconstruction and motion estimation in spatiotemporal imaging, SIAM Journal on Imaging Sciences, 12 (2019), 1686-1719. doi: 10.1137/18M1234047.
[10]	C. Chen, Y. Liu, P. Schniter, M. Tong, K. Zareba, O. Simonetti, L. Potter and R. Ahmad, Ocmr (v1.0)–open-access multi-coil k-space dataset for cardiovascular magnetic resonance imaging, 2020. arXiv: 2008.03410.
[11]	R. K. G. Do, H. Rusinek and B. Taouli, Dynamic contrast-enhanced mr imaging of the liver: Current status and future directions, Magnetic Resonance Imaging Clinics of North America, 17 (2009), 339-349.
[12]	L. Feng, M. B. Srichai, R. P. Lim, A. Harrison, W. King, G. Adluru, E. V. R. Dibella, D. K. Sodickson, R. Otazo and D. Kim, Highly accelerated real-time cardiac cine mri using k–t sparse-sense, Magnetic Resonance in Medicine, 70 (2013), 64-74. doi: 10.1002/mrm.24440.
[13]	J. A. Fessler and D. C. Noll, Iterative image reconstruction in mri with separate magnitude and phase regularization, In 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821), 1 (2004), 209-212.
[14]	H. Gao, S. Rapacchi, D. Wang, J. Moriarty, C. Meehan, J. Sayre, G. Laub, P. Finn and P. Hu, Compressed sensing using prior rank, intensity and sparsity model (prism): Applications in cardiac cine mri, In Proceedings of the 20th Annual Meeting of ISMRM, Melbourne, Australia, 2242 (2012).
[15]	Y. Gordon, S. Partovi and M. Müller-Eschner, et al., Dynamic contrast-enhanced magnetic resonance imaging: Fundamentals and application to the evaluation of the peripheral perfusion, Cardiovascular Diagnosis and Therapy, 4 (2014), 147-164.
[16]	A. Hauptmann, O. Öktem and C. Schönlieb, Image reconstruction in dynamic inverse problems with temporal models, Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging, 2021, 1-31. doi: 10.1007/978-3-030-03009-4_83-1.
[17]	J. S. Jørgensen, E. Ametova, G. Burca, G. Fardell, E. Papoutsellis, E. Pasca, K. Thielemans, M. Turner, R. Warr, W. R. B. Lionheart and P. J. Withers, Core imaging library - part ⅰ: a versatile python framework for tomographic imaging, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 379 (2021), 20200192.
[18]	D. Liang, B. Liu, J. Wang and L. Ying, Accelerating sense using compressed sensing, Magnetic Resonance in Medicine, 62 (2009), 1574-1584. doi: 10.1002/mrm.22161.
[19]	M. Lustig, D. Donoho and J. M. Pauly, Sparse mri: The application of compressed sensing for rapid mr imaging, Magnetic Resonance in Medicine, 58 (2007), 1182-1195.
[20]	R. Otazo, E. Candès and D. K. Sodickson, Low-rank plus sparse matrix decomposition for accelerated dynamic mri with separation of background and dynamic components, Magnetic Resonance in Medicine, 73 (2015), 1125-1136. doi: 10.1002/mrm.25240.
[21]	E. Ovtchinnikov, R. Brown, C. Kolbitsch, E. Pasca, C. da Costa-Luis, A. G. Gillman, B. A. Thomas, N. Efthimiou, J. Mayer, P. Wadhwa, M. J. Ehrhardt, S. Ellis, J. S. Jørgensen, J. Matthews, C. Prieto, A. J. Reader, C. Tsoumpas, M. Turner, D. Atkinson and K. Thielemans, Sirf: Synergistic image reconstruction framework, Computer Physics Communications, 249 (2020), 107087.
[22]	T. Pock, D. Cremers, H. Bischof and A. Chambolle, Global solutions of variational models with convex regularization, SIAM Journal on Imaging Sciences, 3 (2010), 1122-1145. doi: 10.1137/090757617.
[23]	C. Qin, J. Schlemper, J. Caballero, A. N. Price, J. V. Hajnal and D. Rueckert, Convolutional recurrent neural networks for dynamic mr image reconstruction, IEEE Transactions on Medical Imaging, 38 (2019), 280-290. doi: 10.1109/TMI.2018.2863670.
[24]	J. Schlemper, J. Caballero, J. V. Hajnal, A. N. Price and D. Rueckert, A deep cascade of convolutional neural networks for dynamic MR image reconstruction, IEEE Trans. Medical Imaging, 37 (2018), 491-503. doi: 10.1109/TMI.2017.2760978.
[25]	B. Schmitzer, K. P. Schäfers and B. Wirth, Dynamic cell imaging in pet with optimal transport regularization, IEEE Transactions on Medical Imaging, 39 (2020), 1626-1635. doi: 10.1109/TMI.2019.2953773.
[26]	T. Schuster, B. Hahn and M. Burger, Dynamic inverse problems: Modelling—regularization—numerics, Inverse Problems, 34 (2018), 040301, 4pp. doi: 10.1088/1361-6420/aab0f5.
[27]	Z. Stankovic, B. D. Allen, J. Garcia, K. B. Jarvis and M. Markl, 4d flow imaging with mri, Cardiovascular Diagnosis and Therapy, 4 (2014).
[28]	S. Sudarski, T. Henzler and H. Haubenreisser, et al., Free-breathing sparse sampling cine mr imaging with iterative reconstruction for the assessment of left ventricular function and mass at 3.0 t, Radiology, 282 (2017), 74-83. doi: 10.1148/radiol.2016151002.
[29]	P. Tseng, Convergence of a block coordinate descent method for nondifferentiable minimization, Journal of Optimization Theory and Applications, 109 (2001), 475-494. doi: 10.1023/A:1017501703105.
[30]	M. Usman, D. Atkinson, F. Odille, C. Kolbitsch, G. Vaillant, T. Schaeffter, P. G. Batchelor and C. Prieto, Motion corrected compressed sensing for free-breathing dynamic cardiac mri, Magnetic Resonance in Medicine, 70 (2013), 504-516. doi: 10.1002/mrm.24463.
[31]	T. Valkonen, A primal–dual hybrid gradient method for nonlinear operators with applications to mri, Inverse Problems, 30 (2014), 055012, 45pp. doi: 10.1088/0266-5611/30/5/055012.
[32]	Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, Image quality assessment: From error visibility to structural similarity, IEEE Transactions on Image Processing, 13 (2004), 600-612.
[33]	J. Yang, T. Küstner and P. Hu, et al., End-to-end deep learning of non-rigid groupwise registration and reconstruction of dynamic mri, Frontiers in Cardiovascular Medicine, 9 (2022). doi: 10.3389/fcvm.2022.880186.
[34]	N. Zhao, D. O'Connor and B. Adrian, et al., Motion compensated dynamic mri reconstruction with local affine optical flow estimation, EEE Transactions on Bio-Medical Engineering, 66 (2019), 3050-3059. doi: 10.1109/TBME.2019.2900037.