American Institute of Mathematical Sciences

Advanced
December  2020, 7(2): 469-487. doi: 10.3934/jcd.2020019

Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother

Mojtaba F. Fathi 1,, , Ahmadreza Baghaie 2, , Ali Bakhshinejad 3, , Raphael H. Sacho 3, and  Roshan M. D'Souza 1,

1. 

Department of Mechanical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI, USA
2. 

Department of Electrical and Computer Engineering, New York Institute of Technology, Long Island, NY, USA
3. 

Departments of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA

* Corresponding author: Mojtaba F. Fathi

Received  September 2019 Published  July 2020

In this research, we investigate the application of Dynamic Mode Decomposition combined with Kalman Filtering, Smoothing, and Wavelet Denoising (DMD-KF-W) for denoising time-resolved data. We also compare the performance of this technique with state-of-the-art denoising methods such as Total Variation Diminishing (TV) and Divergence-Free Wavelets (DFW), when applicable. Dynamic Mode Decomposition (DMD) is a data-driven method for finding the spatio-temporal structures in time series data. In this research, we use an autoregressive linear model resulting from applying DMD to the time-resolved data as a predictor in a Kalman Filtering-Smoothing framework for the purpose of denoising. The DMD-KF-W method is parameter-free and runs autonomously. Tests on numerical phantoms show lower error metrics when compared to TV and DFW, when applicable. In addition, DMD-KF-W runs an order of magnitude faster than DFW and TV. In the case of synthetic datasets, where the noise-free datasets were available, our method was shown to perform better than TV and DFW methods (when applicable) in terms of the defined error metric.

Keywords: Data driven, dynamic mode decomposition, model order reduction, Kalman Filter, denoising.
Mathematics Subject Classification: Primary: 58F15, 58F17; Secondary: 53C35.
Citation: Mojtaba F. Fathi, Ahmadreza Baghaie, Ali Bakhshinejad, Raphael H. Sacho, Roshan M. D'Souza. Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother. Journal of Computational Dynamics, 2020, 7 (2) : 469-487. doi: 10.3934/jcd.2020019
References:
[1]

E. Bostan, S. Lefkimmiatis, O. Vardoulis, N. Stergiopulos and M. Unser, Improved variational denoising of flow fields with application to phase-contrast MRI data, IEEE Signal Processing Letters, 22 (2015), 762-766.  doi: 10.1109/LSP.2014.2369212.

[2]

K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci., 22 (2012), 887-915.  doi: 10.1007/s00332-012-9130-9.

[3]

Y. Chen, A. Wiesel and A. O. Hero, Shrinkage Estimation of High Dimensional Covariance Matrices, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 2937–2940.

[4]

R. R. Coifman and D. L. Donoho, Translation-invariant de-noising, in Wavelets and Statistics, Springer, (1995), 125–150.

[5]

J. L. Crassidis and J. L. Junkins, Optimal Estimation of Dynamic Systems, Chapman and Hall/CRC, 2004. doi: 10.1201/9780203509128.

[6]

S. T. M. Dawson, M. S. Hemati, M. O. Williams and C. W. Rowley, Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition, Experiments in Fluids, 57 (2016), 1-19.  doi: 10.1007/s00348-016-2127-7.

[7]

M. F. Fathi, A. Bakhshinejad, A. Baghaie and R. M. D'Souza, Dynamic denoising and gappy data reconstruction based on dynamic mode decomposition and discrete cosine transform, Applied Sciences, 8 (2018), 1515. doi: 10.3390/app8091515.

[8]

M. F. Fathi, A. Bakhshinejad, A. Baghaie, D. Saloner, R. H. Sacho, V. L. Rayz and R. M. D'Souza, Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and Lasso regularization, Computerized Medical Imaging and Graphics, 70 (2018), 165-172.  doi: 10.1016/j.compmedimag.2018.07.003.

[9]

M. Gavish and D. L. Donoho, The optimal hard threshold for singular values is $4/\sqrt 3$, IEEE Trans. Inform. Theory, 60 (2014), 5040-5053.  doi: 10.1109/TIT.2014.2323359.

[10] A. Gelb, Applied Optimal Estimation, MIT press, 1974. 
[11]

G. V. Iungo, C. Santoni-Ortiz, M. Abkar, F. Porté-Agel, M. A. Rotea and S. Leonardi, Data-driven reduced order model for prediction of wind turbine wakes, Journal of Physics: Conference Series, 625. doi: 10.1088/1742-6596/625/1/012009.

[12]

K. M. Johnson and M. Markl, Improved SNR in phase contrast velocimetry with five-point balanced flow encoding, Magnetic Resonance in Medicine, 63 (2010), 349-355.  doi: 10.1002/mrm.22202.

[13]

R. E. Kalman, A new approach to linear filtering and prediction problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960), 35-45.  doi: 10.1115/1.3662552.

[14]

J. N. Kutz, S. L. Brunton, B. W. Brunton and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, 2016. doi: 10.1137/1.9781611974508.

[15]

T. Nonomura, H. Shibata and R. Takaki, Dynamic mode decomposition using a kalman filter for parameter estimation, AIP Advances, 8 (2018), 105106. doi: 10.1063/1.5031816.

[16]

T. Nonomura, H. Shibata and R. Takaki, Extended-kalman-filter-based dynamic mode decomposition for simultaneous system identification and denoising, PloS one, 14, e0209836. doi: 10.1371/journal.pone.0209836.

[17]

F. Ong, M. Uecker, U. Tariq, A. Hsiao, M. T. Alley, S. S. Vasanawala and M. Lustig, Robust 4D flow denoising using divergence-free wavelet transform, Magnetic Resonance in Medicine, 73 (2015), 828-842.  doi: 10.1002/mrm.25176.

[18]

H. E. Rauch, F. Tung and C. T. Striebel, Maximum likelihood estimates of linear dynamic systems, AIAA J., 3 (1965), 1445-1450.  doi: 10.2514/3.3166.

[19]

C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), 115-127.  doi: 10.1017/S0022112009992059.

[20]

P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), 5-28.  doi: 10.1017/S0022112010001217.

[21]

P. J. Schmid and J. Sesterhenn, Dynamic mode decomposition of numerical and experimental data, Bulletin of the American Physical Society, 53 (2008), 2008.

[22]

J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421.  doi: 10.3934/jcd.2014.1.391.

[23]

E. Wan, Sigma-point filters: An overview with applications to integrated navigation and vision assisted control, in 2006 IEEE Nonlinear Statistical Signal Processing Workshop, (2006), 201–202. doi: 10.1109/NSSPW.2006.4378854.

[24]

K. Willcox and J. Peraire, Balanced model reduction via the proper orthogonal decomposition, AIAA Journal, 40 (2002), 2323-2330.  doi: 10.2514/6.2001-2611.

[25]

M. O. Williams, C. W. Rowley and I. G. Kevrekidis, A kernel-based method for data-driven Koopman spectral analysis, J. Comput. Dyn., 2 (2015), 247–265. arXiv: 1411.2260. doi: 10.3934/jcd.2015005.

[26]

M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25 (2015), 1307-1346.  doi: 10.1007/s00332-015-9258-5.

[27]

M. J. Zimoń, J. M. Reese and D. R. Emerson, A novel coupling of noise reduction algorithms for particle flow simulations, J. Comput. Phys., 321 (2016), 169-190.  doi: 10.1016/j.jcp.2016.05.049.

show all references

References:
[1]

E. Bostan, S. Lefkimmiatis, O. Vardoulis, N. Stergiopulos and M. Unser, Improved variational denoising of flow fields with application to phase-contrast MRI data, IEEE Signal Processing Letters, 22 (2015), 762-766.  doi: 10.1109/LSP.2014.2369212.

[2]

K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci., 22 (2012), 887-915.  doi: 10.1007/s00332-012-9130-9.

[3]

Y. Chen, A. Wiesel and A. O. Hero, Shrinkage Estimation of High Dimensional Covariance Matrices, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 2937–2940.

[4]

R. R. Coifman and D. L. Donoho, Translation-invariant de-noising, in Wavelets and Statistics, Springer, (1995), 125–150.

[5]

J. L. Crassidis and J. L. Junkins, Optimal Estimation of Dynamic Systems, Chapman and Hall/CRC, 2004. doi: 10.1201/9780203509128.

[6]

S. T. M. Dawson, M. S. Hemati, M. O. Williams and C. W. Rowley, Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition, Experiments in Fluids, 57 (2016), 1-19.  doi: 10.1007/s00348-016-2127-7.

[7]

M. F. Fathi, A. Bakhshinejad, A. Baghaie and R. M. D'Souza, Dynamic denoising and gappy data reconstruction based on dynamic mode decomposition and discrete cosine transform, Applied Sciences, 8 (2018), 1515. doi: 10.3390/app8091515.

[8]

M. F. Fathi, A. Bakhshinejad, A. Baghaie, D. Saloner, R. H. Sacho, V. L. Rayz and R. M. D'Souza, Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and Lasso regularization, Computerized Medical Imaging and Graphics, 70 (2018), 165-172.  doi: 10.1016/j.compmedimag.2018.07.003.

[9]

M. Gavish and D. L. Donoho, The optimal hard threshold for singular values is $4/\sqrt 3$, IEEE Trans. Inform. Theory, 60 (2014), 5040-5053.  doi: 10.1109/TIT.2014.2323359.

[10] A. Gelb, Applied Optimal Estimation, MIT press, 1974. 
[11]

G. V. Iungo, C. Santoni-Ortiz, M. Abkar, F. Porté-Agel, M. A. Rotea and S. Leonardi, Data-driven reduced order model for prediction of wind turbine wakes, Journal of Physics: Conference Series, 625. doi: 10.1088/1742-6596/625/1/012009.

[12]

K. M. Johnson and M. Markl, Improved SNR in phase contrast velocimetry with five-point balanced flow encoding, Magnetic Resonance in Medicine, 63 (2010), 349-355.  doi: 10.1002/mrm.22202.

[13]

R. E. Kalman, A new approach to linear filtering and prediction problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960), 35-45.  doi: 10.1115/1.3662552.

[14]

J. N. Kutz, S. L. Brunton, B. W. Brunton and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, 2016. doi: 10.1137/1.9781611974508.

[15]

T. Nonomura, H. Shibata and R. Takaki, Dynamic mode decomposition using a kalman filter for parameter estimation, AIP Advances, 8 (2018), 105106. doi: 10.1063/1.5031816.

[16]

T. Nonomura, H. Shibata and R. Takaki, Extended-kalman-filter-based dynamic mode decomposition for simultaneous system identification and denoising, PloS one, 14, e0209836. doi: 10.1371/journal.pone.0209836.

[17]

F. Ong, M. Uecker, U. Tariq, A. Hsiao, M. T. Alley, S. S. Vasanawala and M. Lustig, Robust 4D flow denoising using divergence-free wavelet transform, Magnetic Resonance in Medicine, 73 (2015), 828-842.  doi: 10.1002/mrm.25176.

[18]

H. E. Rauch, F. Tung and C. T. Striebel, Maximum likelihood estimates of linear dynamic systems, AIAA J., 3 (1965), 1445-1450.  doi: 10.2514/3.3166.

[19]

C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), 115-127.  doi: 10.1017/S0022112009992059.

[20]

P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), 5-28.  doi: 10.1017/S0022112010001217.

[21]

P. J. Schmid and J. Sesterhenn, Dynamic mode decomposition of numerical and experimental data, Bulletin of the American Physical Society, 53 (2008), 2008.

[22]

J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421.  doi: 10.3934/jcd.2014.1.391.

[23]

E. Wan, Sigma-point filters: An overview with applications to integrated navigation and vision assisted control, in 2006 IEEE Nonlinear Statistical Signal Processing Workshop, (2006), 201–202. doi: 10.1109/NSSPW.2006.4378854.

[24]

K. Willcox and J. Peraire, Balanced model reduction via the proper orthogonal decomposition, AIAA Journal, 40 (2002), 2323-2330.  doi: 10.2514/6.2001-2611.

[25]

M. O. Williams, C. W. Rowley and I. G. Kevrekidis, A kernel-based method for data-driven Koopman spectral analysis, J. Comput. Dyn., 2 (2015), 247–265. arXiv: 1411.2260. doi: 10.3934/jcd.2015005.

[26]

M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25 (2015), 1307-1346.  doi: 10.1007/s00332-015-9258-5.

[27]

M. J. Zimoń, J. M. Reese and D. R. Emerson, A novel coupling of noise reduction algorithms for particle flow simulations, J. Comput. Phys., 321 (2016), 169-190.  doi: 10.1016/j.jcp.2016.05.049.

Figure 1.  The results of the unforced Duffing equation dataset. On the left, the noise-free, noisy, and reconstructed datasets are shown. The numbers at the top of the columns show the corresponding snapshot numbers. The two top rows show the noise-free (reference) and noisy datasets, respectively. The reconstructions of TV and DMD-KF-W methods are shown as indicated. The numbers inside the parenthesis are the overall RMSE values, where the lower is better. On the right, per-snapshot (solid lines) and overall (dotted) RMSE values of the noisy and the reconstructed datasets are presented
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  The results of second test case (wake behind a cylinder at Re = 100). On the left, the reference, noisy, and reconstructed datasets are shown. The images show the snapshot number 21. The bottom row shows the reconstruction errors with the overall RMSE values, where the lower is better. On the right, per-snapshot (solid lines) and overall (dotted) RMSE values of the noisy and the reconstructed datasets are presented
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  The effects of the tuning parameters of TV and DFW on the reconstruction error for the second test dataset. For TV (on left), the effect of the weighting parameter of the regularization term is shown where $ \lambda = 0.07 $ resulted in the best reconstruction. For DFW (on right), the effects of the smallest wavelet level size and the number of cycles are shown. In this case, 5 cycles with the smallest wavelet level size of 10 resulted in the best reconstruction
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  The effects of tuning parameters of TV and DFW on the reconstruction error for the synthetic 4D-Flow MRI dataset. For TV (on left), the effect of the weighting parameter of the regularization term is shown where $ \lambda = 0.03 $ resulted in the best reconstruction. For DFW (on right), the effects of the smallest wavelet level size and the number of cycles are shown. In this case, 4 cycles with the smallest wavelet level size of 12 resulted in the best reconstruction
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  The results for the synthetic 4D-Flow MRI dataset. The synthetic data resembles a cardiac cycle. (a) The top row shows sample slices where at each point, the norm of the velocity is shown. The bottom row shows the norm of difference between the datasets and the noise-free reference. The bottom row figures are titled with the corresponding total RMSE values. All areas outside the geometry mask are zeroed before RMSE values are calculated. (b) Per-snapshot (solid lines) and total (dotted) RMSE values of the noisy and the reconstructed datasets
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  The results for the two in-vivo 4D-Flow MRI datasets. For both datasets, the binary mask is used to exclude all points outside the vessel. The spatial resolutions of datasets (a) and (b) are $ 30 \times 40 \times 103 $ and $ 22 \times 30 \times 48 $, respectively
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7.  The effect of changing the type of Daubechies wavelet used for denoising on the RMSE value
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 8.  Evaluating the effectiveness of wavelet denoising step of the proposed method in improving the RMSE values for the synthetic test cases. The horizontal axis shows the test case number. The vertical axis shows the ratio of the RMSE of reconstruction to the RMSE of the noisy input dataset in percent. A lower ratio corresponds to better denoising
Figure Options

Download full-size image

Download as PowerPoint slide

[1]

Jonathan H. Tu, Clarence W. Rowley, Dirk M. Luchtenburg, Steven L. Brunton, J. Nathan Kutz. On dynamic mode decomposition: Theory and applications. Journal of Computational Dynamics, 2014, 1 (2) : 391-421. doi: 10.3934/jcd.2014.1.391
[2]

Steven L. Brunton, Joshua L. Proctor, Jonathan H. Tu, J. Nathan Kutz. Compressed sensing and dynamic mode decomposition. Journal of Computational Dynamics, 2015, 2 (2) : 165-191. doi: 10.3934/jcd.2015002
[3]

Hao Zhang, Scott T. M. Dawson, Clarence W. Rowley, Eric A. Deem, Louis N. Cattafesta. Evaluating the accuracy of the dynamic mode decomposition. Journal of Computational Dynamics, 2020, 7 (1) : 35-56. doi: 10.3934/jcd.2020002
[4]

Mustaffa Alfatlawi, Vaibhav Srivastava. An incremental approach to online dynamic mode decomposition for time-varying systems with applications to EEG data modeling. Journal of Computational Dynamics, 2020, 7 (2) : 209-241. doi: 10.3934/jcd.2020009
[5]

Junyoung Jang, Kihoon Jang, Hee-Dae Kwon, Jeehyun Lee. Feedback control of an HBV model based on ensemble kalman filter and differential evolution. Mathematical Biosciences & Engineering, 2018, 15 (3) : 667-691. doi: 10.3934/mbe.2018030
[6]

Martin Redmann, Melina A. Freitag. Balanced model order reduction for linear random dynamical systems driven by Lévy noise. Journal of Computational Dynamics, 2018, 5 (1&2) : 33-59. doi: 10.3934/jcd.2018002
[7]

Alexander Bibov, Heikki Haario, Antti Solonen. Stabilized BFGS approximate Kalman filter. Inverse Problems and Imaging, 2015, 9 (4) : 1003-1024. doi: 10.3934/ipi.2015.9.1003
[8]

Russell Johnson, Carmen Núñez. The Kalman-Bucy filter revisited. Discrete and Continuous Dynamical Systems, 2014, 34 (10) : 4139-4153. doi: 10.3934/dcds.2014.34.4139
[9]

Siwei Yu, Jianwei Ma, Stanley Osher. Geometric mode decomposition. Inverse Problems and Imaging, 2018, 12 (4) : 831-852. doi: 10.3934/ipi.2018035
[10]

Wawan Hafid Syaifudin, Endah R. M. Putri. The application of model predictive control on stock portfolio optimization with prediction based on Geometric Brownian Motion-Kalman Filter. Journal of Industrial and Management Optimization, 2021  doi: 10.3934/jimo.2021119
[11]

Andreas Bock, Colin J. Cotter. Learning landmark geodesics using the ensemble Kalman filter. Foundations of Data Science, 2021, 3 (4) : 701-727. doi: 10.3934/fods.2021020
[12]

Amine Laghrib, Lekbir Afraites, Aissam Hadri, Mourad Nachaoui. A non-convex PDE-constrained denoising model for impulse and Gaussian noise mixture reduction. Inverse Problems and Imaging, , () : -. doi: 10.3934/ipi.2022031
[13]

Meixin Xiong, Liuhong Chen, Ju Ming, Jaemin Shin. Accelerating the Bayesian inference of inverse problems by using data-driven compressive sensing method based on proper orthogonal decomposition. Electronic Research Archive, 2021, 29 (5) : 3383-3403. doi: 10.3934/era.2021044
[14]

Andrei Korobeinikov, Aleksei Archibasov, Vladimir Sobolev. Order reduction for an RNA virus evolution model. Mathematical Biosciences & Engineering, 2015, 12 (5) : 1007-1016. doi: 10.3934/mbe.2015.12.1007
[15]

Alberto Bressan, Marco Mazzola, Hongxu Wei. A dynamic model of the limit order book. Discrete and Continuous Dynamical Systems - B, 2020, 25 (3) : 1015-1041. doi: 10.3934/dcdsb.2019206
[16]

Heikki Haario, Leonid Kalachev, Marko Laine. Reduction and identification of dynamic models. Simple example: Generic receptor model. Discrete and Continuous Dynamical Systems - B, 2013, 18 (2) : 417-435. doi: 10.3934/dcdsb.2013.18.417
[17]

Martin Redmann, Peter Benner. Approximation and model order reduction for second order systems with Levy-noise. Conference Publications, 2015, 2015 (special) : 945-953. doi: 10.3934/proc.2015.0945
[18]

A Voutilainen, Jari P. Kaipio. Model reduction and pollution source identification from remote sensing data. Inverse Problems and Imaging, 2009, 3 (4) : 711-730. doi: 10.3934/ipi.2009.3.711
[19]

Vladimir Djordjevic, Vladimir Stojanovic, Hongfeng Tao, Xiaona Song, Shuping He, Weinan Gao. Data-driven control of hydraulic servo actuator based on adaptive dynamic programming. Discrete and Continuous Dynamical Systems - S, 2022, 15 (7) : 1633-1650. doi: 10.3934/dcdss.2021145
[20]

Michael Herty, Adrian Fazekas, Giuseppe Visconti. A two-dimensional data-driven model for traffic flow on highways. Networks and Heterogeneous Media, 2018, 13 (2) : 217-240. doi: 10.3934/nhm.2018010

 Impact Factor: 

Tools

Metrics

  • PDF downloads (669)
  • HTML views (625)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy