American Institute of Mathematical Sciences

Advanced
December  2020, 7(2): 469-487. doi: 10.3934/jcd.2020019

Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother

Mojtaba F. Fathi 1,, , Ahmadreza Baghaie 2, , Ali Bakhshinejad 3, , Raphael H. Sacho 3, and  Roshan M. D'Souza 1,

1. 

Department of Mechanical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI, USA
2. 

Department of Electrical and Computer Engineering, New York Institute of Technology, Long Island, NY, USA
3. 

Departments of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA

* Corresponding author: Mojtaba F. Fathi

Received  September 2019 Published  July 2020

In this research, we investigate the application of Dynamic Mode Decomposition combined with Kalman Filtering, Smoothing, and Wavelet Denoising (DMD-KF-W) for denoising time-resolved data. We also compare the performance of this technique with state-of-the-art denoising methods such as Total Variation Diminishing (TV) and Divergence-Free Wavelets (DFW), when applicable. Dynamic Mode Decomposition (DMD) is a data-driven method for finding the spatio-temporal structures in time series data. In this research, we use an autoregressive linear model resulting from applying DMD to the time-resolved data as a predictor in a Kalman Filtering-Smoothing framework for the purpose of denoising. The DMD-KF-W method is parameter-free and runs autonomously. Tests on numerical phantoms show lower error metrics when compared to TV and DFW, when applicable. In addition, DMD-KF-W runs an order of magnitude faster than DFW and TV. In the case of synthetic datasets, where the noise-free datasets were available, our method was shown to perform better than TV and DFW methods (when applicable) in terms of the defined error metric.

Keywords: Data driven, dynamic mode decomposition, model order reduction, Kalman Filter, denoising.
Mathematics Subject Classification: Primary: 58F15, 58F17; Secondary: 53C35.
Citation: Mojtaba F. Fathi, Ahmadreza Baghaie, Ali Bakhshinejad, Raphael H. Sacho, Roshan M. D'Souza. Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother. Journal of Computational Dynamics, 2020, 7 (2) : 469-487. doi: 10.3934/jcd.2020019
References:
[1]

E. Bostan, S. Lefkimmiatis, O. Vardoulis, N. Stergiopulos and M. Unser, Improved variational denoising of flow fields with application to phase-contrast MRI data, IEEE Signal Processing Letters, 22 (2015), 762-766.  doi: 10.1109/LSP.2014.2369212.  Google Scholar

[2]

K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci., 22 (2012), 887-915.  doi: 10.1007/s00332-012-9130-9.  Google Scholar

[3]

Y. Chen, A. Wiesel and A. O. Hero, Shrinkage Estimation of High Dimensional Covariance Matrices, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 2937–2940. Google Scholar

[4]

R. R. Coifman and D. L. Donoho, Translation-invariant de-noising, in Wavelets and Statistics, Springer, (1995), 125–150. Google Scholar

[5]

J. L. Crassidis and J. L. Junkins, Optimal Estimation of Dynamic Systems, Chapman and Hall/CRC, 2004. doi: 10.1201/9780203509128.  Google Scholar

[6]

S. T. M. Dawson, M. S. Hemati, M. O. Williams and C. W. Rowley, Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition, Experiments in Fluids, 57 (2016), 1-19.  doi: 10.1007/s00348-016-2127-7.  Google Scholar

[7]

M. F. Fathi, A. Bakhshinejad, A. Baghaie and R. M. D'Souza, Dynamic denoising and gappy data reconstruction based on dynamic mode decomposition and discrete cosine transform, Applied Sciences, 8 (2018), 1515. doi: 10.3390/app8091515.  Google Scholar

[8]

M. F. Fathi, A. Bakhshinejad, A. Baghaie, D. Saloner, R. H. Sacho, V. L. Rayz and R. M. D'Souza, Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and Lasso regularization, Computerized Medical Imaging and Graphics, 70 (2018), 165-172.  doi: 10.1016/j.compmedimag.2018.07.003.  Google Scholar

[9]

M. Gavish and D. L. Donoho, The optimal hard threshold for singular values is $4/\sqrt 3$, IEEE Trans. Inform. Theory, 60 (2014), 5040-5053.  doi: 10.1109/TIT.2014.2323359.  Google Scholar

[10] A. Gelb, Applied Optimal Estimation, MIT press, 1974.   Google Scholar
[11]

G. V. Iungo, C. Santoni-Ortiz, M. Abkar, F. Porté-Agel, M. A. Rotea and S. Leonardi, Data-driven reduced order model for prediction of wind turbine wakes, Journal of Physics: Conference Series, 625. doi: 10.1088/1742-6596/625/1/012009.  Google Scholar

[12]

K. M. Johnson and M. Markl, Improved SNR in phase contrast velocimetry with five-point balanced flow encoding, Magnetic Resonance in Medicine, 63 (2010), 349-355.  doi: 10.1002/mrm.22202.  Google Scholar

[13]

R. E. Kalman, A new approach to linear filtering and prediction problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960), 35-45.  doi: 10.1115/1.3662552.  Google Scholar

[14]

J. N. Kutz, S. L. Brunton, B. W. Brunton and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, 2016. doi: 10.1137/1.9781611974508.  Google Scholar

[15]

T. Nonomura, H. Shibata and R. Takaki, Dynamic mode decomposition using a kalman filter for parameter estimation, AIP Advances, 8 (2018), 105106. doi: 10.1063/1.5031816.  Google Scholar

[16]

T. Nonomura, H. Shibata and R. Takaki, Extended-kalman-filter-based dynamic mode decomposition for simultaneous system identification and denoising, PloS one, 14, e0209836. doi: 10.1371/journal.pone.0209836.  Google Scholar

[17]

F. Ong, M. Uecker, U. Tariq, A. Hsiao, M. T. Alley, S. S. Vasanawala and M. Lustig, Robust 4D flow denoising using divergence-free wavelet transform, Magnetic Resonance in Medicine, 73 (2015), 828-842.  doi: 10.1002/mrm.25176.  Google Scholar

[18]

H. E. Rauch, F. Tung and C. T. Striebel, Maximum likelihood estimates of linear dynamic systems, AIAA J., 3 (1965), 1445-1450.  doi: 10.2514/3.3166.  Google Scholar

[19]

C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), 115-127.  doi: 10.1017/S0022112009992059.  Google Scholar

[20]

P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), 5-28.  doi: 10.1017/S0022112010001217.  Google Scholar

[21]

P. J. Schmid and J. Sesterhenn, Dynamic mode decomposition of numerical and experimental data, Bulletin of the American Physical Society, 53 (2008), 2008. Google Scholar

[22]

J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421.  doi: 10.3934/jcd.2014.1.391.  Google Scholar

[23]

E. Wan, Sigma-point filters: An overview with applications to integrated navigation and vision assisted control, in 2006 IEEE Nonlinear Statistical Signal Processing Workshop, (2006), 201–202. doi: 10.1109/NSSPW.2006.4378854.  Google Scholar

[24]

K. Willcox and J. Peraire, Balanced model reduction via the proper orthogonal decomposition, AIAA Journal, 40 (2002), 2323-2330.  doi: 10.2514/6.2001-2611.  Google Scholar

[25]

M. O. Williams, C. W. Rowley and I. G. Kevrekidis, A kernel-based method for data-driven Koopman spectral analysis, J. Comput. Dyn., 2 (2015), 247–265. arXiv: 1411.2260. doi: 10.3934/jcd.2015005.  Google Scholar

[26]

M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25 (2015), 1307-1346.  doi: 10.1007/s00332-015-9258-5.  Google Scholar

[27]

M. J. Zimoń, J. M. Reese and D. R. Emerson, A novel coupling of noise reduction algorithms for particle flow simulations, J. Comput. Phys., 321 (2016), 169-190.  doi: 10.1016/j.jcp.2016.05.049.  Google Scholar

show all references

References:
[1]

E. Bostan, S. Lefkimmiatis, O. Vardoulis, N. Stergiopulos and M. Unser, Improved variational denoising of flow fields with application to phase-contrast MRI data, IEEE Signal Processing Letters, 22 (2015), 762-766.  doi: 10.1109/LSP.2014.2369212.  Google Scholar

[2]

K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci., 22 (2012), 887-915.  doi: 10.1007/s00332-012-9130-9.  Google Scholar

[3]

Y. Chen, A. Wiesel and A. O. Hero, Shrinkage Estimation of High Dimensional Covariance Matrices, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 2937–2940. Google Scholar

[4]

R. R. Coifman and D. L. Donoho, Translation-invariant de-noising, in Wavelets and Statistics, Springer, (1995), 125–150. Google Scholar

[5]

J. L. Crassidis and J. L. Junkins, Optimal Estimation of Dynamic Systems, Chapman and Hall/CRC, 2004. doi: 10.1201/9780203509128.  Google Scholar

[6]

S. T. M. Dawson, M. S. Hemati, M. O. Williams and C. W. Rowley, Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition, Experiments in Fluids, 57 (2016), 1-19.  doi: 10.1007/s00348-016-2127-7.  Google Scholar

[7]

M. F. Fathi, A. Bakhshinejad, A. Baghaie and R. M. D'Souza, Dynamic denoising and gappy data reconstruction based on dynamic mode decomposition and discrete cosine transform, Applied Sciences, 8 (2018), 1515. doi: 10.3390/app8091515.  Google Scholar

[8]

M. F. Fathi, A. Bakhshinejad, A. Baghaie, D. Saloner, R. H. Sacho, V. L. Rayz and R. M. D'Souza, Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and Lasso regularization, Computerized Medical Imaging and Graphics, 70 (2018), 165-172.  doi: 10.1016/j.compmedimag.2018.07.003.  Google Scholar

[9]

M. Gavish and D. L. Donoho, The optimal hard threshold for singular values is $4/\sqrt 3$, IEEE Trans. Inform. Theory, 60 (2014), 5040-5053.  doi: 10.1109/TIT.2014.2323359.  Google Scholar

[10] A. Gelb, Applied Optimal Estimation, MIT press, 1974.   Google Scholar
[11]

G. V. Iungo, C. Santoni-Ortiz, M. Abkar, F. Porté-Agel, M. A. Rotea and S. Leonardi, Data-driven reduced order model for prediction of wind turbine wakes, Journal of Physics: Conference Series, 625. doi: 10.1088/1742-6596/625/1/012009.  Google Scholar

[12]

K. M. Johnson and M. Markl, Improved SNR in phase contrast velocimetry with five-point balanced flow encoding, Magnetic Resonance in Medicine, 63 (2010), 349-355.  doi: 10.1002/mrm.22202.  Google Scholar

[13]

R. E. Kalman, A new approach to linear filtering and prediction problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960), 35-45.  doi: 10.1115/1.3662552.  Google Scholar

[14]

J. N. Kutz, S. L. Brunton, B. W. Brunton and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, 2016. doi: 10.1137/1.9781611974508.  Google Scholar

[15]

T. Nonomura, H. Shibata and R. Takaki, Dynamic mode decomposition using a kalman filter for parameter estimation, AIP Advances, 8 (2018), 105106. doi: 10.1063/1.5031816.  Google Scholar

[16]

T. Nonomura, H. Shibata and R. Takaki, Extended-kalman-filter-based dynamic mode decomposition for simultaneous system identification and denoising, PloS one, 14, e0209836. doi: 10.1371/journal.pone.0209836.  Google Scholar

[17]

F. Ong, M. Uecker, U. Tariq, A. Hsiao, M. T. Alley, S. S. Vasanawala and M. Lustig, Robust 4D flow denoising using divergence-free wavelet transform, Magnetic Resonance in Medicine, 73 (2015), 828-842.  doi: 10.1002/mrm.25176.  Google Scholar

[18]

H. E. Rauch, F. Tung and C. T. Striebel, Maximum likelihood estimates of linear dynamic systems, AIAA J., 3 (1965), 1445-1450.  doi: 10.2514/3.3166.  Google Scholar

[19]

C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), 115-127.  doi: 10.1017/S0022112009992059.  Google Scholar

[20]

P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), 5-28.  doi: 10.1017/S0022112010001217.  Google Scholar

[21]

P. J. Schmid and J. Sesterhenn, Dynamic mode decomposition of numerical and experimental data, Bulletin of the American Physical Society, 53 (2008), 2008. Google Scholar

[22]

J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421.  doi: 10.3934/jcd.2014.1.391.  Google Scholar

[23]

E. Wan, Sigma-point filters: An overview with applications to integrated navigation and vision assisted control, in 2006 IEEE Nonlinear Statistical Signal Processing Workshop, (2006), 201–202. doi: 10.1109/NSSPW.2006.4378854.  Google Scholar

[24]

K. Willcox and J. Peraire, Balanced model reduction via the proper orthogonal decomposition, AIAA Journal, 40 (2002), 2323-2330.  doi: 10.2514/6.2001-2611.  Google Scholar

[25]

M. O. Williams, C. W. Rowley and I. G. Kevrekidis, A kernel-based method for data-driven Koopman spectral analysis, J. Comput. Dyn., 2 (2015), 247–265. arXiv: 1411.2260. doi: 10.3934/jcd.2015005.  Google Scholar

[26]

M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci., 25 (2015), 1307-1346.  doi: 10.1007/s00332-015-9258-5.  Google Scholar

[27]

M. J. Zimoń, J. M. Reese and D. R. Emerson, A novel coupling of noise reduction algorithms for particle flow simulations, J. Comput. Phys., 321 (2016), 169-190.  doi: 10.1016/j.jcp.2016.05.049.  Google Scholar

Figure 1.  The results of the unforced Duffing equation dataset. On the left, the noise-free, noisy, and reconstructed datasets are shown. The numbers at the top of the columns show the corresponding snapshot numbers. The two top rows show the noise-free (reference) and noisy datasets, respectively. The reconstructions of TV and DMD-KF-W methods are shown as indicated. The numbers inside the parenthesis are the overall RMSE values, where the lower is better. On the right, per-snapshot (solid lines) and overall (dotted) RMSE values of the noisy and the reconstructed datasets are presented
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  The results of second test case (wake behind a cylinder at Re = 100). On the left, the reference, noisy, and reconstructed datasets are shown. The images show the snapshot number 21. The bottom row shows the reconstruction errors with the overall RMSE values, where the lower is better. On the right, per-snapshot (solid lines) and overall (dotted) RMSE values of the noisy and the reconstructed datasets are presented
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  The effects of the tuning parameters of TV and DFW on the reconstruction error for the second test dataset. For TV (on left), the effect of the weighting parameter of the regularization term is shown where $ \lambda = 0.07 $ resulted in the best reconstruction. For DFW (on right), the effects of the smallest wavelet level size and the number of cycles are shown. In this case, 5 cycles with the smallest wavelet level size of 10 resulted in the best reconstruction
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  The effects of tuning parameters of TV and DFW on the reconstruction error for the synthetic 4D-Flow MRI dataset. For TV (on left), the effect of the weighting parameter of the regularization term is shown where $ \lambda = 0.03 $ resulted in the best reconstruction. For DFW (on right), the effects of the smallest wavelet level size and the number of cycles are shown. In this case, 4 cycles with the smallest wavelet level size of 12 resulted in the best reconstruction
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  The results for the synthetic 4D-Flow MRI dataset. The synthetic data resembles a cardiac cycle. (a) The top row shows sample slices where at each point, the norm of the velocity is shown. The bottom row shows the norm of difference between the datasets and the noise-free reference. The bottom row figures are titled with the corresponding total RMSE values. All areas outside the geometry mask are zeroed before RMSE values are calculated. (b) Per-snapshot (solid lines) and total (dotted) RMSE values of the noisy and the reconstructed datasets
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  The results for the two in-vivo 4D-Flow MRI datasets. For both datasets, the binary mask is used to exclude all points outside the vessel. The spatial resolutions of datasets (a) and (b) are $ 30 \times 40 \times 103 $ and $ 22 \times 30 \times 48 $, respectively
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7.  The effect of changing the type of Daubechies wavelet used for denoising on the RMSE value
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 8.  Evaluating the effectiveness of wavelet denoising step of the proposed method in improving the RMSE values for the synthetic test cases. The horizontal axis shows the test case number. The vertical axis shows the ratio of the RMSE of reconstruction to the RMSE of the noisy input dataset in percent. A lower ratio corresponds to better denoising
Figure Options

Download full-size image

Download as PowerPoint slide

[1]

Abdelghafour Atlas, Mostafa Bendahmane, Fahd Karami, Driss Meskine, Omar Oubbih. A nonlinear fractional reaction-diffusion system applied to image denoising and decomposition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020321
[2]

Shuang Chen, Jinqiao Duan, Ji Li. Effective reduction of a three-dimensional circadian oscillator model. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020349
[3]

Dan Zhu, Rosemary A. Renaut, Hongwei Li, Tianyou Liu. Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory. Inverse Problems & Imaging, 2021, 15 (1) : 159-183. doi: 10.3934/ipi.2020076
[4]

Kevin Li. Dynamic transitions of the Swift-Hohenberg equation with third-order dispersion. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2021003
[5]

Tien-Yu Lin, Bhaba R. Sarker, Chien-Jui Lin. An optimal setup cost reduction and lot size for economic production quantity model with imperfect quality and quantity discounts. Journal of Industrial & Management Optimization, 2021, 17 (1) : 467-484. doi: 10.3934/jimo.2020043
[6]

Van Duong Dinh. Random data theory for the cubic fourth-order nonlinear Schrödinger equation. Communications on Pure & Applied Analysis, 2021, 20 (2) : 651-680. doi: 10.3934/cpaa.2020284
[7]

Chao Xing, Jiaojiao Pan, Hong Luo. Stability and dynamic transition of a toxin-producing phytoplankton-zooplankton model with additional food. Communications on Pure & Applied Analysis, 2021, 20 (1) : 427-448. doi: 10.3934/cpaa.2020275
[8]

Mahdi Karimi, Seyed Jafar Sadjadi. Optimization of a Multi-Item Inventory model for deteriorating items with capacity constraint using dynamic programming. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2021013
[9]

Musen Xue, Guowei Zhu. Partial myopia vs. forward-looking behaviors in a dynamic pricing and replenishment model for perishable items. Journal of Industrial & Management Optimization, 2021, 17 (2) : 633-648. doi: 10.3934/jimo.2019126
[10]

Liupeng Wang, Yunqing Huang. Error estimates for second-order SAV finite element method to phase field crystal model. Electronic Research Archive, 2021, 29 (1) : 1735-1752. doi: 10.3934/era.2020089
[11]

Xinpeng Wang, Bingo Wing-Kuen Ling, Wei-Chao Kuang, Zhijing Yang. Orthogonal intrinsic mode functions via optimization approach. Journal of Industrial & Management Optimization, 2021, 17 (1) : 51-66. doi: 10.3934/jimo.2019098
[12]

Javier Fernández, Cora Tori, Marcela Zuccalli. Lagrangian reduction of nonholonomic discrete mechanical systems by stages. Journal of Geometric Mechanics, 2020, 12 (4) : 607-639. doi: 10.3934/jgm.2020029
[13]

Yujuan Li, Huaifu Wang, Peipei Zhou, Guoshuang Zhang. Some properties of the cycle decomposition of WG-NLFSR. Advances in Mathematics of Communications, 2021, 15 (1) : 155-165. doi: 10.3934/amc.2020050
[14]

Zhiyan Ding, Qin Li, Jianfeng Lu. Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds. Foundations of Data Science, 2020  doi: 10.3934/fods.2020018
[15]

Håkon Hoel, Gaukhar Shaimerdenova, Raúl Tempone. Multilevel Ensemble Kalman Filtering based on a sample average of independent EnKF estimators. Foundations of Data Science, 2020, 2 (4) : 351-390. doi: 10.3934/fods.2020017
[16]

Elvio Accinelli, Humberto Muñiz. A dynamic for production economies with multiple equilibria. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021002
[17]

Zonghong Cao, Jie Min. Selection and impact of decision mode of encroachment and retail service in a dual-channel supply chain. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020167
[18]

Pierluigi Colli, Gianni Gilardi, Gabriela Marinoschi. Solvability and sliding mode control for the viscous Cahn–Hilliard system with a possibly singular potential. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020051
[19]

Emre Esentürk, Juan Velazquez. Large time behavior of exchange-driven growth. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 747-775. doi: 10.3934/dcds.2020299
[20]

Chao Xing, Zhigang Pan, Quan Wang. Stabilities and dynamic transitions of the Fitzhugh-Nagumo system. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 775-794. doi: 10.3934/dcdsb.2020134

 Impact Factor: 

Tools

Metrics

  • PDF downloads (205)
  • HTML views (233)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences