American Institute of Mathematical Sciences

Advanced
June  2015, 2(2): 247-265. doi: 10.3934/jcd.2015005

A kernel-based method for data-driven koopman spectral analysis

Matthew O. Williams 1, , Clarence W. Rowley 2, and  Ioannis G. Kevrekidis 3,

1. 

United Technologies Research Center, 411 Silver Lane, East Hartford, CT 06118, United States
2. 

Dept. of Mechanical and Aerospace Engineering, Princeton University, Princeton, NJ 08544
3. 

Department of Chemical and Biological Engineering & PACM, Princeton University, Princeton, NJ 08544, United States

Received  February 2015 Revised  February 2016 Published  May 2016

A data-driven, kernel-based method for approximating the leading Koopman eigenvalues, eigenfunctions, and modes in problems with high-dimensional state spaces is presented. This approach uses a set of scalar observables (functions that map a state to a scalar value) that are defined implicitly by the feature map associated with a user-defined kernel function. This circumvents the computational issues that arise due to the number of functions required to span a ``sufficiently rich'' subspace of all possible scalar observables in such applications. We illustrate this method on two examples: the first is the FitzHugh-Nagumo PDE, a prototypical one-dimensional reaction-diffusion system, and the second is a set of vorticity data computed from experimentally obtained velocity data from flow past a cylinder at Reynolds number 413. In both examples, we use the output of Dynamic Mode Decomposition, which has a similar computational cost, as the benchmark for our approach.
Keywords: time series analysis, machine learning., kernel methods, Koopman operator, dynamic mode decomposition.
Mathematics Subject Classification: Primary: 37M10, 65P99, 47B3.
Citation: Matthew O. Williams, Clarence W. Rowley, Ioannis G. Kevrekidis. A kernel-based method for data-driven koopman spectral analysis. Journal of Computational Dynamics, 2015, 2 (2) : 247-265. doi: 10.3934/jcd.2015005
References:
[1]

S. Bagheri, Koopman-mode decomposition of the cylinder wake, Journal of Fluid Mechanics, 726 (2013), 596-623. doi: 10.1017/jfm.2013.249.  Google Scholar

[2]

S. Bagheri, Effects of weak noise on oscillating flows: Linking quality factor, Floquet modes, and Koopman spectrum, Physics of Fluids, 26 (2014), 094104. doi: 10.1063/1.4895898.  Google Scholar

[3]

G. Baudat and F. Anouar, Kernel-based methods and function approximation, In Proceedings of the International Joint Conference on Neural Networks, IEEE, 2 (2001), 1244-1249. doi: 10.1109/IJCNN.2001.939539.  Google Scholar

[4]

C. M. Bishop et al, Pattern Recognition and Machine Learning, Springer, 2006. doi: 10.1007/978-0-387-45528-0.  Google Scholar

[5]

J. P. Boyd, Chebyshev and Fourier Spectral Methods, Courier Dover Publications, Mineola, NY, 2001.  Google Scholar

[6]

M. Budišić, R. Mohr and I. Mezić, Applied Koopmanism, Chaos: An Interdisciplinary Journal of Nonlinear Science, 22 (2012), 047510, 33pp. doi: 10.1063/1.4772195.  Google Scholar

[7]

C. J. Burges, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, 2 (1998), 121-167. Google Scholar

[8]

A. Chatterjee, An introduction to the proper orthogonal decomposition, Current Science, 78 (2000), 808-817. Google Scholar

[9]

K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, Journal of Nonlinear Science, 22 (2012), 887-915. doi: 10.1007/s00332-012-9130-9.  Google Scholar

[10]

R. R. Coifman and S. Lafon, Diffusion maps, Applied and Computational Harmonic Analysis, 21 (2006), 5-30. doi: 10.1016/j.acha.2006.04.006.  Google Scholar

[11]

N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University Press, 2000. doi: 10.1017/CBO9780511801389.  Google Scholar

[12]

C. E. Elmer and E. S. Van Vleck, Spatially discrete FitzHugh-Nagumo equations, SIAM Journal on Applied Mathematics, 65 (2005), 1153-1174. doi: 10.1137/S003613990343687X.  Google Scholar

[13]

P. Gaspard and S. Tasaki, Liouvillian dynamics of the {Hopf} bifurcation, Physical Review E, 64 (2001), 056232, 17pp. doi: 10.1103/PhysRevE.64.056232.  Google Scholar

[14]

M. S. Hemati, M. O. Williams and C. W. Rowley, Dynamic mode decomposition for large and streaming datasets, Physics of Fluids, 26 (2014), 111701. doi: 10.1063/1.4901016.  Google Scholar

[15]

P. Holmes, J. L. Lumley, G. Berkooz and C. W. Rowley, Turbulence, Coherent Structures, Dynamical Systems and Symmetry, Cambridge University Press, 2nd edition, 2012. doi: 10.1017/CBO9780511919701.  Google Scholar

[16]

M. R. Jovanović, P. J. Schmid and J. W. Nichols, Sparsity-promoting dynamic mode decomposition, Physics of Fluids, 26 (2014), 024103. Google Scholar

[17]

J.-N. Juang, Applied System Identification, Prentice Hall, 1994. Google Scholar

[18]

B. O. Koopman and J. von Neumann, Dynamical systems of continuous spectra, Proceedings of the National Academy of Sciences of the United States of America, 18 (1932), 255-163. doi: 10.1073/pnas.18.3.255.  Google Scholar

[19]

B. O. Koopman, Hamiltonian systems and transformation in Hilbert space, Proceedings of the National Academy of Sciences of the United States of America, 17 (1931), 315-318. doi: 10.1073/pnas.17.5.315.  Google Scholar

[20]

J. A. Lee and M. Verleysen, Nonlinear Dimensionality Reduction, Springer, 2007. doi: 10.1007/978-0-387-39351-3.  Google Scholar

[21]

R. B. Lehoucq, D. C. Sorensen and C. Yang, ARPACK Users' Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods, volume 6. SIAM, 1998. doi: 10.1137/1.9780898719628.  Google Scholar

[22]

A. Mauroy and I. Mezic, A spectral operator-theoretic framework for global stability, In 52nd IEEE Conference on Decision and Control, (2013), 5234-5239. doi: 10.1109/CDC.2013.6760712.  Google Scholar

[23]

I. Mezić, Spectral properties of dynamical systems, model reduction and decompositions, Nonlinear Dynamics, 41 (2005), 309-325. doi: 10.1007/s11071-005-2824-x.  Google Scholar

[24]

I. Mezić, Analysis of fluid flows via spectral properties of the Koopman operator, Annual Review of Fluid Mechanics, 45 (2013), 357-378. doi: 10.1146/annurev-fluid-011212-140652.  Google Scholar

[25]

C. E. Rasmussen, Gaussian Processes for Machine Learning, MIT Press, 2006.  Google Scholar

[26]

C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, Journal of Fluid Mechanics, 641 (2009), 115-127. doi: 10.1017/S0022112009992059.  Google Scholar

[27]

P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, Journal of Fluid Mechanics, 656 (2010), 5-28. doi: 10.1017/S0022112010001217.  Google Scholar

[28]

P. J. Schmid, D. Violato and F. Scarano, Decomposition of time-resolved tomographic PIV, Experiments in Fluids, 52 (2012), 1567-1579. doi: 10.1007/s00348-012-1266-8.  Google Scholar

[29]

P. J. Schmid, L. Li, M. P. Juniper and O. Pust, Applications of the dynamic mode decomposition, Theoretical and Computational Fluid Dynamics, 25 (2011), 249-259. doi: 10.1007/s00162-010-0203-9.  Google Scholar

[30]

B. Scholkopf, The kernel trick for distances, Advances in Neural Information Processing Systems, (2001), 301-307. Google Scholar

[31]

L. Sirovich, Turbulence and the dynamics of coherent structures. part I: Coherent structures, Quarterly of applied mathematics, 45 (1987), 561-571.  Google Scholar

[32]

G. Tissot, L. Cordier, N. Benard and B. R. Noack, Model reduction using dynamic mode decomposition, Comptes Rendus Mćcanique, 342 (2014), 410-416. doi: 10.1016/j.crme.2013.12.011.  Google Scholar

[33]

J. H. Tu, C. W. Rowley, J. N. Kutz and J. K. Shang, Toward compressed DMD: Spectral analysis of fluid flows using sub-Nyquist-rate PIV data, Experiments in Fluids, 55 (2014), 1-13. Google Scholar

[34]

J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421. doi: 10.3934/jcd.2014.1.391.  Google Scholar

[35]

M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, Journal of Nonlinear Science, 25 (2015), 1307-1346. doi: 10.1007/s00332-015-9258-5.  Google Scholar

[36]

A. Wynn, D. S. Pearson, B. Ganapathisubramani and P. J. Goulart, Optimal mode decomposition for unsteady flows, Journal of Fluid Mechanics, 733 (2013), 473-503. doi: 10.1017/jfm.2013.426.  Google Scholar

show all references

References:
[1]

S. Bagheri, Koopman-mode decomposition of the cylinder wake, Journal of Fluid Mechanics, 726 (2013), 596-623. doi: 10.1017/jfm.2013.249.  Google Scholar

[2]

S. Bagheri, Effects of weak noise on oscillating flows: Linking quality factor, Floquet modes, and Koopman spectrum, Physics of Fluids, 26 (2014), 094104. doi: 10.1063/1.4895898.  Google Scholar

[3]

G. Baudat and F. Anouar, Kernel-based methods and function approximation, In Proceedings of the International Joint Conference on Neural Networks, IEEE, 2 (2001), 1244-1249. doi: 10.1109/IJCNN.2001.939539.  Google Scholar

[4]

C. M. Bishop et al, Pattern Recognition and Machine Learning, Springer, 2006. doi: 10.1007/978-0-387-45528-0.  Google Scholar

[5]

J. P. Boyd, Chebyshev and Fourier Spectral Methods, Courier Dover Publications, Mineola, NY, 2001.  Google Scholar

[6]

M. Budišić, R. Mohr and I. Mezić, Applied Koopmanism, Chaos: An Interdisciplinary Journal of Nonlinear Science, 22 (2012), 047510, 33pp. doi: 10.1063/1.4772195.  Google Scholar

[7]

C. J. Burges, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, 2 (1998), 121-167. Google Scholar

[8]

A. Chatterjee, An introduction to the proper orthogonal decomposition, Current Science, 78 (2000), 808-817. Google Scholar

[9]

K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, Journal of Nonlinear Science, 22 (2012), 887-915. doi: 10.1007/s00332-012-9130-9.  Google Scholar

[10]

R. R. Coifman and S. Lafon, Diffusion maps, Applied and Computational Harmonic Analysis, 21 (2006), 5-30. doi: 10.1016/j.acha.2006.04.006.  Google Scholar

[11]

N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University Press, 2000. doi: 10.1017/CBO9780511801389.  Google Scholar

[12]

C. E. Elmer and E. S. Van Vleck, Spatially discrete FitzHugh-Nagumo equations, SIAM Journal on Applied Mathematics, 65 (2005), 1153-1174. doi: 10.1137/S003613990343687X.  Google Scholar

[13]

P. Gaspard and S. Tasaki, Liouvillian dynamics of the {Hopf} bifurcation, Physical Review E, 64 (2001), 056232, 17pp. doi: 10.1103/PhysRevE.64.056232.  Google Scholar

[14]

M. S. Hemati, M. O. Williams and C. W. Rowley, Dynamic mode decomposition for large and streaming datasets, Physics of Fluids, 26 (2014), 111701. doi: 10.1063/1.4901016.  Google Scholar

[15]

P. Holmes, J. L. Lumley, G. Berkooz and C. W. Rowley, Turbulence, Coherent Structures, Dynamical Systems and Symmetry, Cambridge University Press, 2nd edition, 2012. doi: 10.1017/CBO9780511919701.  Google Scholar

[16]

M. R. Jovanović, P. J. Schmid and J. W. Nichols, Sparsity-promoting dynamic mode decomposition, Physics of Fluids, 26 (2014), 024103. Google Scholar

[17]

J.-N. Juang, Applied System Identification, Prentice Hall, 1994. Google Scholar

[18]

B. O. Koopman and J. von Neumann, Dynamical systems of continuous spectra, Proceedings of the National Academy of Sciences of the United States of America, 18 (1932), 255-163. doi: 10.1073/pnas.18.3.255.  Google Scholar

[19]

B. O. Koopman, Hamiltonian systems and transformation in Hilbert space, Proceedings of the National Academy of Sciences of the United States of America, 17 (1931), 315-318. doi: 10.1073/pnas.17.5.315.  Google Scholar

[20]

J. A. Lee and M. Verleysen, Nonlinear Dimensionality Reduction, Springer, 2007. doi: 10.1007/978-0-387-39351-3.  Google Scholar

[21]

R. B. Lehoucq, D. C. Sorensen and C. Yang, ARPACK Users' Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods, volume 6. SIAM, 1998. doi: 10.1137/1.9780898719628.  Google Scholar

[22]

A. Mauroy and I. Mezic, A spectral operator-theoretic framework for global stability, In 52nd IEEE Conference on Decision and Control, (2013), 5234-5239. doi: 10.1109/CDC.2013.6760712.  Google Scholar

[23]

I. Mezić, Spectral properties of dynamical systems, model reduction and decompositions, Nonlinear Dynamics, 41 (2005), 309-325. doi: 10.1007/s11071-005-2824-x.  Google Scholar

[24]

I. Mezić, Analysis of fluid flows via spectral properties of the Koopman operator, Annual Review of Fluid Mechanics, 45 (2013), 357-378. doi: 10.1146/annurev-fluid-011212-140652.  Google Scholar

[25]

C. E. Rasmussen, Gaussian Processes for Machine Learning, MIT Press, 2006.  Google Scholar

[26]

C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, Journal of Fluid Mechanics, 641 (2009), 115-127. doi: 10.1017/S0022112009992059.  Google Scholar

[27]

P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, Journal of Fluid Mechanics, 656 (2010), 5-28. doi: 10.1017/S0022112010001217.  Google Scholar

[28]

P. J. Schmid, D. Violato and F. Scarano, Decomposition of time-resolved tomographic PIV, Experiments in Fluids, 52 (2012), 1567-1579. doi: 10.1007/s00348-012-1266-8.  Google Scholar

[29]

P. J. Schmid, L. Li, M. P. Juniper and O. Pust, Applications of the dynamic mode decomposition, Theoretical and Computational Fluid Dynamics, 25 (2011), 249-259. doi: 10.1007/s00162-010-0203-9.  Google Scholar

[30]

B. Scholkopf, The kernel trick for distances, Advances in Neural Information Processing Systems, (2001), 301-307. Google Scholar

[31]

L. Sirovich, Turbulence and the dynamics of coherent structures. part I: Coherent structures, Quarterly of applied mathematics, 45 (1987), 561-571.  Google Scholar

[32]

G. Tissot, L. Cordier, N. Benard and B. R. Noack, Model reduction using dynamic mode decomposition, Comptes Rendus Mćcanique, 342 (2014), 410-416. doi: 10.1016/j.crme.2013.12.011.  Google Scholar

[33]

J. H. Tu, C. W. Rowley, J. N. Kutz and J. K. Shang, Toward compressed DMD: Spectral analysis of fluid flows using sub-Nyquist-rate PIV data, Experiments in Fluids, 55 (2014), 1-13. Google Scholar

[34]

J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421. doi: 10.3934/jcd.2014.1.391.  Google Scholar

[35]

M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, Journal of Nonlinear Science, 25 (2015), 1307-1346. doi: 10.1007/s00332-015-9258-5.  Google Scholar

[36]

A. Wynn, D. S. Pearson, B. Ganapathisubramani and P. J. Goulart, Optimal mode decomposition for unsteady flows, Journal of Fluid Mechanics, 733 (2013), 473-503. doi: 10.1017/jfm.2013.426.  Google Scholar

[1]

Jonathan H. Tu, Clarence W. Rowley, Dirk M. Luchtenburg, Steven L. Brunton, J. Nathan Kutz. On dynamic mode decomposition: Theory and applications. Journal of Computational Dynamics, 2014, 1 (2) : 391-421. doi: 10.3934/jcd.2014.1.391
[2]

Steven L. Brunton, Joshua L. Proctor, Jonathan H. Tu, J. Nathan Kutz. Compressed sensing and dynamic mode decomposition. Journal of Computational Dynamics, 2015, 2 (2) : 165-191. doi: 10.3934/jcd.2015002
[3]

Hao Zhang, Scott T. M. Dawson, Clarence W. Rowley, Eric A. Deem, Louis N. Cattafesta. Evaluating the accuracy of the dynamic mode decomposition. Journal of Computational Dynamics, 2020, 7 (1) : 35-56. doi: 10.3934/jcd.2020002
[4]

Mojtaba F. Fathi, Ahmadreza Baghaie, Ali Bakhshinejad, Raphael H. Sacho, Roshan M. D'Souza. Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother. Journal of Computational Dynamics, 2020, 7 (2) : 469-487. doi: 10.3934/jcd.2020019
[5]

Mustaffa Alfatlawi, Vaibhav Srivastava. An incremental approach to online dynamic mode decomposition for time-varying systems with applications to EEG data modeling. Journal of Computational Dynamics, 2020, 7 (2) : 209-241. doi: 10.3934/jcd.2020009
[6]

Stefan Klus, Christof Schütte. Towards tensor-based methods for the numerical approximation of the Perron--Frobenius and Koopman operator. Journal of Computational Dynamics, 2016, 3 (2) : 139-161. doi: 10.3934/jcd.2016007
[7]

Jiang Xie, Junfu Xu, Celine Nie, Qing Nie. Machine learning of swimming data via wisdom of crowd and regression analysis. Mathematical Biosciences & Engineering, 2017, 14 (2) : 511-527. doi: 10.3934/mbe.2017031
[8]

Andreas Chirstmann, Qiang Wu, Ding-Xuan Zhou. Preface to the special issue on analysis in machine learning and data science. Communications on Pure & Applied Analysis, 2020, 19 (8) : i-iii. doi: 10.3934/cpaa.2020171
[9]

Siwei Yu, Jianwei Ma, Stanley Osher. Geometric mode decomposition. Inverse Problems & Imaging, 2018, 12 (4) : 831-852. doi: 10.3934/ipi.2018035
[10]

Jae-Hong Pyo, Jie Shen. Normal mode analysis of second-order projection methods for incompressible flows. Discrete & Continuous Dynamical Systems - B, 2005, 5 (3) : 817-840. doi: 10.3934/dcdsb.2005.5.817
[11]

Stefan Klus, Péter Koltai, Christof Schütte. On the numerical approximation of the Perron-Frobenius and Koopman operator. Journal of Computational Dynamics, 2016, 3 (1) : 51-79. doi: 10.3934/jcd.2016003
[12]

Yu-Ting Lin, John Malik, Hau-Tieng Wu. Wave-shape oscillatory model for nonstationary periodic time series analysis. Foundations of Data Science, 2021, 3 (2) : 99-131. doi: 10.3934/fods.2021009
[13]

Ping Yan, Ji-Bo Wang, Li-Qiang Zhao. Single-machine bi-criterion scheduling with release times and exponentially time-dependent learning effects. Journal of Industrial & Management Optimization, 2019, 15 (3) : 1117-1131. doi: 10.3934/jimo.2018088
[14]

Xingong Zhang. Single machine and flowshop scheduling problems with sum-of-processing time based learning phenomenon. Journal of Industrial & Management Optimization, 2020, 16 (1) : 231-244. doi: 10.3934/jimo.2018148
[15]

Rakesh Pilkar, Erik M. Bollt, Charles Robinson. Empirical mode decomposition/Hilbert transform analysis of postural responses to small amplitude anterior-posterior sinusoidal translations of varying frequencies. Mathematical Biosciences & Engineering, 2011, 8 (4) : 1085-1097. doi: 10.3934/mbe.2011.8.1085
[16]

Ahmad Deeb, A. Hamdouni, Dina Razafindralandy. Comparison between Borel-Padé summation and factorial series, as time integration methods. Discrete & Continuous Dynamical Systems - S, 2016, 9 (2) : 393-408. doi: 10.3934/dcdss.2016003
[17]

Mickaël D. Chekroun, Jean Roux. Homeomorphisms group of normed vector space: Conjugacy problems and the Koopman operator. Discrete & Continuous Dynamical Systems, 2013, 33 (9) : 3957-3980. doi: 10.3934/dcds.2013.33.3957
[18]

Émilie Chouzenoux, Henri Gérard, Jean-Christophe Pesquet. General risk measures for robust machine learning. Foundations of Data Science, 2019, 1 (3) : 249-269. doi: 10.3934/fods.2019011
[19]

Ana Rita Nogueira, João Gama, Carlos Abreu Ferreira. Causal discovery in machine learning: Theories and applications. Journal of Dynamics & Games, 2021, 8 (3) : 203-231. doi: 10.3934/jdg.2021008
[20]

Annalisa Pascarella, Alberto Sorrentino, Cristina Campi, Michele Piana. Particle filtering, beamforming and multiple signal classification for the analysis of magnetoencephalography time series: a comparison of algorithms. Inverse Problems & Imaging, 2010, 4 (1) : 169-190. doi: 10.3934/ipi.2010.4.169

 Impact Factor: 

Tools

Metrics

  • PDF downloads (643)
  • HTML views (0)
  • Cited by (64)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy