American Institute of Mathematical Sciences

Advanced
December  2020, 7(2): 469-487. doi: 10.3934/jcd.2020019

Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother

Mojtaba F. Fathi 1,, , Ahmadreza Baghaie 2, , Ali Bakhshinejad 3, , Raphael H. Sacho 3, and  Roshan M. D'Souza 1,

1. 

Department of Mechanical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI, USA
2. 

Department of Electrical and Computer Engineering, New York Institute of Technology, Long Island, NY, USA
3. 

Departments of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA

* Corresponding author: Mojtaba F. Fathi

Received  September 2019 Published  July 2020

In this research, we investigate the application of Dynamic Mode Decomposition combined with Kalman Filtering, Smoothing, and Wavelet Denoising (DMD-KF-W) for denoising time-resolved data. We also compare the performance of this technique with state-of-the-art denoising methods such as Total Variation Diminishing (TV) and Divergence-Free Wavelets (DFW), when applicable. Dynamic Mode Decomposition (DMD) is a data-driven method for finding the spatio-temporal structures in time series data. In this research, we use an autoregressive linear model resulting from applying DMD to the time-resolved data as a predictor in a Kalman Filtering-Smoothing framework for the purpose of denoising. The DMD-KF-W method is parameter-free and runs autonomously. Tests on numerical phantoms show lower error metrics when compared to TV and DFW, when applicable. In addition, DMD-KF-W runs an order of magnitude faster than DFW and TV. In the case of synthetic datasets, where the noise-free datasets were available, our method was shown to perform better than TV and DFW methods (when applicable) in terms of the defined error metric.

Keywords: Data driven, dynamic mode decomposition, model order reduction, Kalman Filter, denoising.
Mathematics Subject Classification: Primary: 58F15, 58F17; Secondary: 53C35.
Citation: Mojtaba F. Fathi, Ahmadreza Baghaie, Ali Bakhshinejad, Raphael H. Sacho, Roshan M. D'Souza. Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother. Journal of Computational Dynamics, 2020, 7 (2) : 469-487. doi: 10.3934/jcd.2020019
References:
[1]

E. Bostan, S. Lefkimmiatis, O. Vardoulis, N. Stergiopulos and M. Unser, Improved variational denoising of flow fields with application to phase-contrast MRI data, IEEE Signal Processing Letters, 22 (2015), 762-766.  doi: 10.1109/LSP.2014.2369212.  Google Scholar

[2]

K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci., 22 (2012), 887-915.  doi: 10.1007/s00332-012-9130-9.  Google Scholar

[3]

Y. Chen, A. Wiesel and A. O. Hero, Shrinkage Estimation of High Dimensional Covariance Matrices, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 2937–2940. Google Scholar

[4]

R. R. Coifman and D. L. Donoho, Translation-invariant de-noising, in Wavelets and Statistics, Springer, (1995), 125–150. Google Scholar

[5]

J. L. Crassidis and J. L. Junkins, Optimal Estimation of Dynamic Systems, Chapman and Hall/CRC, 2004. doi: 10.1201/9780203509128.  Google Scholar

[6]

S. T. M. Dawson, M. S. Hemati, M. O. Williams and C. W. Rowley, Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition, Experiments in Fluids, 57 (2016), 1-19.  doi: 10.1007/s00348-016-2127-7.  Google Scholar

[7]

M. F. Fathi, A. Bakhshinejad, A. Baghaie and R. M. D'Souza, Dynamic denoising and gappy data reconstruction based on dynamic mode decomposition and discrete cosine transform, Applied Sciences, 8 (2018), 1515. doi: 10.3390/app8091515.  Google Scholar

[8]

M. F. Fathi, A. Bakhshinejad, A. Baghaie, D. Saloner, R. H. Sacho, V. L. Rayz and R. M. D'Souza, Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and Lasso regularization, Computerized Medical Imaging and Graphics, 70 (2018), 165-172.  doi: 10.1016/j.compmedimag.2018.07.003.  Google Scholar

[9]

M. Gavish and D. L. Donoho, The optimal hard threshold for singular values is $4/\sqrt 3$, IEEE Trans. Inform. Theory, 60 (2014), 5040-5053.  doi: 10.1109/TIT.2014.2323359.  Google Scholar

[10] A. Gelb, Applied Optimal Estimation, MIT press, 1974.   Google Scholar
[11]

G. V. Iungo, C. Santoni-Ortiz, M. Abkar, F. Porté-Agel, M. A. Rotea and S. Leonardi, Data-driven reduced order model for prediction of wind turbine wakes, Journal of Physics: Conference Series, 625. doi: 10.1088/1742-6596/625/1/012009.  Google Scholar

[12]

K. M. Johnson and M. Markl, Improved SNR in phase contrast velocimetry with five-point balanced flow encoding, Magnetic Resonance in Medicine, 63 (2010), 349-355.  doi: 10.1002/mrm.22202.  Google Scholar

[13]

R. E. Kalman, A new approach to linear filtering and prediction problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960), 35-45.  doi: 10.1115/1.3662552.  Google Scholar

[14]

J. N. Kutz, S. L. Brunton, B. W. Brunton and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, 2016. doi: 10.1137/1.9781611974508.  Google Scholar

[15]

T. Nonomura, H. Shibata and R. Takaki, Dynamic mode decomposition using a kalman filter for parameter estimation, AIP Advances, 8 (2018), 105106. doi: 10.1063/1.5031816.  Google Scholar

[16]

T. Nonomura, H. Shibata and R. Takaki, Extended-kalman-filter-based dynamic mode decomposition for simultaneous system identification and denoising, PloS one, 14, e0209836. doi: 10.1371/journal.pone.0209836.  Google Scholar

[17]

F. Ong, M. Uecker, U. Tariq, A. Hsiao, M. T. Alley, S. S. Vasanawala and M. Lustig, Robust 4D flow denoising using divergence-free wavelet transform, Magnetic Resonance in Medicine, 73 (2015), 828-842.  doi: 10.1002/mrm.25176.  Google Scholar

[18]

H. E. Rauch, F. Tung and C. T. Striebel, Maximum likelihood estimates of linear dynamic systems, AIAA J., 3 (1965), 1445-1450.  doi: 10.2514/3.3166.  Google Scholar

[19]

C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), 115-127.  doi: 10.1017/S0022112009992059.  Google Scholar

[20]

P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), 5-28.  doi: 10.1017/S0022112010001217.  Google Scholar

[21]

P. J. Schmid and J. Sesterhenn, Dynamic mode decomposition of numerical and experimental data, Bulletin of the American Physical Society, 53 (2008), 2008. Google Scholar

[22]

J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421.  doi: 10.3934/jcd.2014.1.391.  Google Scholar

[23]

E. Wan, Sigma-point filters: An overview with applications to integrated navigation and vision assisted control, in 2006 IEEE Nonlinear Statistical Signal Processing Workshop, (2006), 201–202. doi: 10.1109/NSSPW.2006.4378854.  Google Scholar

[24]

K. Willcox and J. Peraire, Balanced model reduction via the proper orthogonal decomposition, AIAA Journal, 40 (2002), 2323-2330.  doi: 10.2514/6.2001-2611.  Google Scholar

[25]

M. O. Williams, C. W. Rowley and I. G. Kevrekidis, A kernel-based method for data-driven Koopman spectral analysis, J. Comput. Dyn., 2 (2015), 247–265. arXiv: 1411.2260. doi: 10.3934/jcd.2015005.  Google Scholar

[26]

M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25 (2015), 1307-1346.  doi: 10.1007/s00332-015-9258-5.  Google Scholar

[27]

M. J. Zimoń, J. M. Reese and D. R. Emerson, A novel coupling of noise reduction algorithms for particle flow simulations, J. Comput. Phys., 321 (2016), 169-190.  doi: 10.1016/j.jcp.2016.05.049.  Google Scholar

show all references

References:
[1]

E. Bostan, S. Lefkimmiatis, O. Vardoulis, N. Stergiopulos and M. Unser, Improved variational denoising of flow fields with application to phase-contrast MRI data, IEEE Signal Processing Letters, 22 (2015), 762-766.  doi: 10.1109/LSP.2014.2369212.  Google Scholar

[2]

K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci., 22 (2012), 887-915.  doi: 10.1007/s00332-012-9130-9.  Google Scholar

[3]

Y. Chen, A. Wiesel and A. O. Hero, Shrinkage Estimation of High Dimensional Covariance Matrices, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 2937–2940. Google Scholar

[4]

R. R. Coifman and D. L. Donoho, Translation-invariant de-noising, in Wavelets and Statistics, Springer, (1995), 125–150. Google Scholar

[5]

J. L. Crassidis and J. L. Junkins, Optimal Estimation of Dynamic Systems, Chapman and Hall/CRC, 2004. doi: 10.1201/9780203509128.  Google Scholar

[6]

S. T. M. Dawson, M. S. Hemati, M. O. Williams and C. W. Rowley, Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition, Experiments in Fluids, 57 (2016), 1-19.  doi: 10.1007/s00348-016-2127-7.  Google Scholar

[7]

M. F. Fathi, A. Bakhshinejad, A. Baghaie and R. M. D'Souza, Dynamic denoising and gappy data reconstruction based on dynamic mode decomposition and discrete cosine transform, Applied Sciences, 8 (2018), 1515. doi: 10.3390/app8091515.  Google Scholar

[8]

M. F. Fathi, A. Bakhshinejad, A. Baghaie, D. Saloner, R. H. Sacho, V. L. Rayz and R. M. D'Souza, Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and Lasso regularization, Computerized Medical Imaging and Graphics, 70 (2018), 165-172.  doi: 10.1016/j.compmedimag.2018.07.003.  Google Scholar

[9]

M. Gavish and D. L. Donoho, The optimal hard threshold for singular values is $4/\sqrt 3$, IEEE Trans. Inform. Theory, 60 (2014), 5040-5053.  doi: 10.1109/TIT.2014.2323359.  Google Scholar

[10] A. Gelb, Applied Optimal Estimation, MIT press, 1974.   Google Scholar
[11]

G. V. Iungo, C. Santoni-Ortiz, M. Abkar, F. Porté-Agel, M. A. Rotea and S. Leonardi, Data-driven reduced order model for prediction of wind turbine wakes, Journal of Physics: Conference Series, 625. doi: 10.1088/1742-6596/625/1/012009.  Google Scholar

[12]

K. M. Johnson and M. Markl, Improved SNR in phase contrast velocimetry with five-point balanced flow encoding, Magnetic Resonance in Medicine, 63 (2010), 349-355.  doi: 10.1002/mrm.22202.  Google Scholar

[13]

R. E. Kalman, A new approach to linear filtering and prediction problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960), 35-45.  doi: 10.1115/1.3662552.  Google Scholar

[14]

J. N. Kutz, S. L. Brunton, B. W. Brunton and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, 2016. doi: 10.1137/1.9781611974508.  Google Scholar

[15]

T. Nonomura, H. Shibata and R. Takaki, Dynamic mode decomposition using a kalman filter for parameter estimation, AIP Advances, 8 (2018), 105106. doi: 10.1063/1.5031816.  Google Scholar

[16]

T. Nonomura, H. Shibata and R. Takaki, Extended-kalman-filter-based dynamic mode decomposition for simultaneous system identification and denoising, PloS one, 14, e0209836. doi: 10.1371/journal.pone.0209836.  Google Scholar

[17]

F. Ong, M. Uecker, U. Tariq, A. Hsiao, M. T. Alley, S. S. Vasanawala and M. Lustig, Robust 4D flow denoising using divergence-free wavelet transform, Magnetic Resonance in Medicine, 73 (2015), 828-842.  doi: 10.1002/mrm.25176.  Google Scholar

[18]

H. E. Rauch, F. Tung and C. T. Striebel, Maximum likelihood estimates of linear dynamic systems, AIAA J., 3 (1965), 1445-1450.  doi: 10.2514/3.3166.  Google Scholar

[19]

C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), 115-127.  doi: 10.1017/S0022112009992059.  Google Scholar

[20]

P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), 5-28.  doi: 10.1017/S0022112010001217.  Google Scholar

[21]

P. J. Schmid and J. Sesterhenn, Dynamic mode decomposition of numerical and experimental data, Bulletin of the American Physical Society, 53 (2008), 2008. Google Scholar

[22]

J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421.  doi: 10.3934/jcd.2014.1.391.  Google Scholar

[23]

E. Wan, Sigma-point filters: An overview with applications to integrated navigation and vision assisted control, in 2006 IEEE Nonlinear Statistical Signal Processing Workshop, (2006), 201–202. doi: 10.1109/NSSPW.2006.4378854.  Google Scholar

[24]

K. Willcox and J. Peraire, Balanced model reduction via the proper orthogonal decomposition, AIAA Journal, 40 (2002), 2323-2330.  doi: 10.2514/6.2001-2611.  Google Scholar

[25]

M. O. Williams, C. W. Rowley and I. G. Kevrekidis, A kernel-based method for data-driven Koopman spectral analysis, J. Comput. Dyn., 2 (2015), 247–265. arXiv: 1411.2260. doi: 10.3934/jcd.2015005.  Google Scholar

[26]

M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25 (2015), 1307-1346.  doi: 10.1007/s00332-015-9258-5.  Google Scholar

[27]

M. J. Zimoń, J. M. Reese and D. R. Emerson, A novel coupling of noise reduction algorithms for particle flow simulations, J. Comput. Phys., 321 (2016), 169-190.  doi: 10.1016/j.jcp.2016.05.049.  Google Scholar

Figure 1.  The results of the unforced Duffing equation dataset. On the left, the noise-free, noisy, and reconstructed datasets are shown. The numbers at the top of the columns show the corresponding snapshot numbers. The two top rows show the noise-free (reference) and noisy datasets, respectively. The reconstructions of TV and DMD-KF-W methods are shown as indicated. The numbers inside the parenthesis are the overall RMSE values, where the lower is better. On the right, per-snapshot (solid lines) and overall (dotted) RMSE values of the noisy and the reconstructed datasets are presented
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  The results of second test case (wake behind a cylinder at Re = 100). On the left, the reference, noisy, and reconstructed datasets are shown. The images show the snapshot number 21. The bottom row shows the reconstruction errors with the overall RMSE values, where the lower is better. On the right, per-snapshot (solid lines) and overall (dotted) RMSE values of the noisy and the reconstructed datasets are presented
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  The effects of the tuning parameters of TV and DFW on the reconstruction error for the second test dataset. For TV (on left), the effect of the weighting parameter of the regularization term is shown where $ \lambda = 0.07 $ resulted in the best reconstruction. For DFW (on right), the effects of the smallest wavelet level size and the number of cycles are shown. In this case, 5 cycles with the smallest wavelet level size of 10 resulted in the best reconstruction
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  The effects of tuning parameters of TV and DFW on the reconstruction error for the synthetic 4D-Flow MRI dataset. For TV (on left), the effect of the weighting parameter of the regularization term is shown where $ \lambda = 0.03 $ resulted in the best reconstruction. For DFW (on right), the effects of the smallest wavelet level size and the number of cycles are shown. In this case, 4 cycles with the smallest wavelet level size of 12 resulted in the best reconstruction
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  The results for the synthetic 4D-Flow MRI dataset. The synthetic data resembles a cardiac cycle. (a) The top row shows sample slices where at each point, the norm of the velocity is shown. The bottom row shows the norm of difference between the datasets and the noise-free reference. The bottom row figures are titled with the corresponding total RMSE values. All areas outside the geometry mask are zeroed before RMSE values are calculated. (b) Per-snapshot (solid lines) and total (dotted) RMSE values of the noisy and the reconstructed datasets
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  The results for the two in-vivo 4D-Flow MRI datasets. For both datasets, the binary mask is used to exclude all points outside the vessel. The spatial resolutions of datasets (a) and (b) are $ 30 \times 40 \times 103 $ and $ 22 \times 30 \times 48 $, respectively
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7.  The effect of changing the type of Daubechies wavelet used for denoising on the RMSE value
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 8.  Evaluating the effectiveness of wavelet denoising step of the proposed method in improving the RMSE values for the synthetic test cases. The horizontal axis shows the test case number. The vertical axis shows the ratio of the RMSE of reconstruction to the RMSE of the noisy input dataset in percent. A lower ratio corresponds to better denoising
Figure Options

Download full-size image

Download as PowerPoint slide

[1]

Jonathan H. Tu, Clarence W. Rowley, Dirk M. Luchtenburg, Steven L. Brunton, J. Nathan Kutz. On dynamic mode decomposition: Theory and applications. Journal of Computational Dynamics, 2014, 1 (2) : 391-421. doi: 10.3934/jcd.2014.1.391
[2]

Steven L. Brunton, Joshua L. Proctor, Jonathan H. Tu, J. Nathan Kutz. Compressed sensing and dynamic mode decomposition. Journal of Computational Dynamics, 2015, 2 (2) : 165-191. doi: 10.3934/jcd.2015002
[3]

Hao Zhang, Scott T. M. Dawson, Clarence W. Rowley, Eric A. Deem, Louis N. Cattafesta. Evaluating the accuracy of the dynamic mode decomposition. Journal of Computational Dynamics, 2020, 7 (1) : 35-56. doi: 10.3934/jcd.2020002
[4]

Mustaffa Alfatlawi, Vaibhav Srivastava. An incremental approach to online dynamic mode decomposition for time-varying systems with applications to EEG data modeling. Journal of Computational Dynamics, 2020, 7 (2) : 209-241. doi: 10.3934/jcd.2020009
[5]

Junyoung Jang, Kihoon Jang, Hee-Dae Kwon, Jeehyun Lee. Feedback control of an HBV model based on ensemble kalman filter and differential evolution. Mathematical Biosciences & Engineering, 2018, 15 (3) : 667-691. doi: 10.3934/mbe.2018030
[6]

Alexander Bibov, Heikki Haario, Antti Solonen. Stabilized BFGS approximate Kalman filter. Inverse Problems & Imaging, 2015, 9 (4) : 1003-1024. doi: 10.3934/ipi.2015.9.1003
[7]

Russell Johnson, Carmen Núñez. The Kalman-Bucy filter revisited. Discrete & Continuous Dynamical Systems, 2014, 34 (10) : 4139-4153. doi: 10.3934/dcds.2014.34.4139
[8]

Martin Redmann, Melina A. Freitag. Balanced model order reduction for linear random dynamical systems driven by Lévy noise. Journal of Computational Dynamics, 2018, 5 (1&2) : 33-59. doi: 10.3934/jcd.2018002
[9]

Siwei Yu, Jianwei Ma, Stanley Osher. Geometric mode decomposition. Inverse Problems & Imaging, 2018, 12 (4) : 831-852. doi: 10.3934/ipi.2018035
[10]

Andrei Korobeinikov, Aleksei Archibasov, Vladimir Sobolev. Order reduction for an RNA virus evolution model. Mathematical Biosciences & Engineering, 2015, 12 (5) : 1007-1016. doi: 10.3934/mbe.2015.12.1007
[11]

Alberto Bressan, Marco Mazzola, Hongxu Wei. A dynamic model of the limit order book. Discrete & Continuous Dynamical Systems - B, 2020, 25 (3) : 1015-1041. doi: 10.3934/dcdsb.2019206
[12]

Heikki Haario, Leonid Kalachev, Marko Laine. Reduction and identification of dynamic models. Simple example: Generic receptor model. Discrete & Continuous Dynamical Systems - B, 2013, 18 (2) : 417-435. doi: 10.3934/dcdsb.2013.18.417
[13]

Martin Redmann, Peter Benner. Approximation and model order reduction for second order systems with Levy-noise. Conference Publications, 2015, 2015 (special) : 945-953. doi: 10.3934/proc.2015.0945
[14]

A Voutilainen, Jari P. Kaipio. Model reduction and pollution source identification from remote sensing data. Inverse Problems & Imaging, 2009, 3 (4) : 711-730. doi: 10.3934/ipi.2009.3.711
[15]

Michael Herty, Adrian Fazekas, Giuseppe Visconti. A two-dimensional data-driven model for traffic flow on highways. Networks & Heterogeneous Media, 2018, 13 (2) : 217-240. doi: 10.3934/nhm.2018010
[16]

Z. G. Feng, Kok Lay Teo, N. U. Ahmed, Yulin Zhao, W. Y. Yan. Optimal fusion of sensor data for Kalman filtering. Discrete & Continuous Dynamical Systems, 2006, 14 (3) : 483-503. doi: 10.3934/dcds.2006.14.483
[17]

Abdelghafour Atlas, Mostafa Bendahmane, Fahd Karami, Driss Meskine, Omar Oubbih. A nonlinear fractional reaction-diffusion system applied to image denoising and decomposition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020321
[18]

Rongliang Chen, Jizu Huang, Xiao-Chuan Cai. A parallel domain decomposition algorithm for large scale image denoising. Inverse Problems & Imaging, 2019, 13 (6) : 1259-1282. doi: 10.3934/ipi.2019055
[19]

Valerii Maltsev, Michael Pokojovy. On a parabolic-hyperbolic filter for multicolor image noise reduction. Evolution Equations & Control Theory, 2016, 5 (2) : 251-272. doi: 10.3934/eect.2016004
[20]

Mohammad Afzalinejad, Zahra Abbasi. A slacks-based model for dynamic data envelopment analysis. Journal of Industrial & Management Optimization, 2019, 15 (1) : 275-291. doi: 10.3934/jimo.2018043

 Impact Factor: 

Tools

Metrics

  • PDF downloads (268)
  • HTML views (238)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences