June  2017, 4(1&2): 1-19. doi: 10.3934/jcd.2017001

Kernel methods for the approximation of some key quantities of nonlinear systems

1. 

Laboratory for Computational and Statistical Learning, Massachusetts Institute of Technology, Cambridge, MA, USA

2. 

Department of Mathematics, AlFaisal University, Riyadh, KSA

* Corresponding author:Boumediene Hamzi

Published  April 2017

Fund Project: BH thanks the European Commission and the Scientific and the Technological Research Council of Turkey (Tubitak) for financial support received through a Marie Curie Fellowship, and JB gratefully acknowledges support under NSF contracts NSF-IIS-08-03293 and NSF-CCF-08-08847 to M. Maggioni.
Parts of this work were done while the authors were at the Department of Mathematics of Duke University and then while the second author was with the Departments of Mathematics of Imperial College London, Yildiz Technical University and Ko¸c University for Marie Curie Fellowships and at the Fields Institute.

We introduce a data-based approach to estimating key quantities which arise in the study of nonlinear control systems and random nonlinear dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems -with a reasonable expectation of success -once the nonlinear system has been mapped into a high or infinite dimensional feature space. In particular, we embed a nonlinear system in a reproducing kernel Hilbert space where linear theory can be used to develop computable, non-parametric estimators approximating controllability and observability energy functions for nonlinear systems. In all cases the relevant quantities are estimated from simulated or observed data. It is then shown that the controllability energy estimator provides a key means for approximating the invariant measure of an ergodic, stochastically forced nonlinear system.

Citation: Jake Bouvrie, Boumediene Hamzi. Kernel methods for the approximation of some key quantities of nonlinear systems. Journal of Computational Dynamics, 2017, 4 (1&2) : 1-19. doi: 10.3934/jcd.2017001
References:
[1]

N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc., 68 (1950), 337-404.  doi: 10.1090/S0002-9947-1950-0051437-7.  Google Scholar

[2]

G. BiauB. Cadre and B. Pelletier, Exact rates in density support estimation, J. Multivariate Anal., 99 (2008), 2185-2207.  doi: 10.1016/j.jmva.2008.02.021.  Google Scholar

[3]

V. I. Bogachev, Gaussian Measures, American Mathematical Society, 1998.  Google Scholar

[4]

J. Bouvrie and B. Hamzi, Balanced reduction of nonlinear control systems in reproducing kernel Hilbert space, in Proc. 48th Annual Allerton Conference on Communication, Control, and Computing, (2010), 294-301, http://arxiv.org/abs/1011.2952. doi: 10.1109/ALLERTON.2010.5706920.  Google Scholar

[5]

J. Bouvrie and B. Hamzi, Kernel methods for the approximation of nonlinear systems, in SIAM Journal on Control and Optimization, (2017), to appear, https://arxiv.org/abs/1108.2903. Google Scholar

[6]

J. Bouvrie and B. Hamzi, Empirical estimators for stochastically forced nonlinear systems: Observability, controllability and the invariant measure, in Proc. American Control Conference (ACC), 2012, (2012). doi: 10.1109/ACC.2012.6315175.  Google Scholar

[7]

R. Brockett, Stochastic Control, Lecture Notes, Harvard University Press, 2009. Google Scholar

[8]

R. L. Butchart, An explicit solution to the Fokker-Planck equation for an ordinary differential equation, Int. J. Control, 1 (1965), 201-208.  doi: 10.1080/00207176508905472.  Google Scholar

[9]

A. Caponnetto and E. De Vito, Optimal rates for the regularized least-squares algorithm, Found. Comput. Math., 7 (2007), 331-368.  doi: 10.1007/s10208-006-0196-8.  Google Scholar

[10]

F. Cucker and S. Smale, On the mathematical foundations of learning, Bull. AMS, 39 (2002), 1-49.  doi: 10.1090/S0273-0979-01-00923-5.  Google Scholar

[11]

G. Da Prato, An Introduction to Infinite Dimensional Analysis, Springer, 2006. doi: 10.1007/3-540-29021-4.  Google Scholar

[12]

G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, Cambridge University Press, 1992. doi: 10.1017/CBO9780511666223.  Google Scholar

[13]

E. De Vito, L. Rosasco and A. Toigo, Spectral Regularization for Support Estimation, in J. Shawe-Taylor et al., eds., Advances in Neural Information Processing Systems (NIPS), 24, Vancouver, Curran Associates, Inc., 2010. Google Scholar

[14]

G. E. Dullerud and F. Paganini, A Course in Robust Control Theory: A Convex Approach, Springer, 2000. doi: 10.1007/978-1-4757-3290-0.  Google Scholar

[15]

G. FroylandK. JuddA. I. MeesK. Murao and D. Watson, Constructing invariant measures from data, Int. J. Bifurcat. Chaos, 5 (1995), 1181-1192.   Google Scholar

[16]

G. Froyland, Extracting dynamical behaviour via Markov models, In Alistair Mees, ed., Nonlinear Dynamics and Statistics: Proceedings, Newton Institute, Cambridge, (2001), 281–321, Birkhauser.  Google Scholar

[17]

K. Fujimoto and D. Tsubakino, Computation of nonlinear balanced realization and model reduction based on Taylor series expansion, Systems and Control Letters, 57 (2008), 283-289.  doi: 10.1016/j.sysconle.2007.08.015.  Google Scholar

[18]

A. T. Fuller, Analysis of nonlinear stochastic systems by means of the Fokker-Planck equation, Int. J. Control, 9 (1969), 603-655.  doi: 10.1007/3-540-29021-4.  Google Scholar

[19]

P. Giesl and H. Wendland, Meshless collocation: Error estimates with application to dynamical systems, SIAM J. Num. Anal., 45 (2007), 1723-1749.  doi: 10.1137/060658813.  Google Scholar

[20]

P. Giesl, Construction of Global Lyapunov Functions Using Radial Basis Functions, Springer, 2007.  Google Scholar

[21]

W. S. Gray and E. I. Verriest, Algebraically defined gramians for nonlinear systems, Proc. of the 45th IEEE CDC , (2006). doi: 10.1109/CDC.2006.376840.  Google Scholar

[22]

J. GuinezR. Quintero and A. D. Rueda, Calculating steady states for a Fokker-Planck equation, Acta Math. Hungar., 91 (2001), 311-323.  doi: 10.1023/A:1010615818034.  Google Scholar

[23]

C. Hartmann and C. Schuette, Balancing of partially-observed stochastic differential equations, Proc. of the 47th IEEE CDC, (2008), 4867-4872.  doi: 10.1007/3-540-29021-4.  Google Scholar

[24]

D. KilminsterD. Allingham and A. Mees, Estimating invariant probability densities for dynamical systems: Nonparametric approach to time series analysis, Ann. Ⅰ. Stat. Math., 39 (2002), 1-49.  doi: 10.1023/A:1016134209348.  Google Scholar

[25]

A. J. Krener, The Important State Coordinates of a Nonlinear System, In Advances in control theory and applications, C. Bonivento, A. Isidori, L. Marconi, C. Rossi, editors, 353 (2007), 161-170, Springer. doi: 10.1007/978-3-540-70701-1_8.  Google Scholar

[26]

A. J. Krener, Reduced order modeling of nonlinear control systems, In Analysis and Design of Nonlinear Control Systems, A. Astolfi and L. Marconi, editors, (2008), 41-62, Springer. doi: 10.1007/978-3-540-74358-3_4.  Google Scholar

[27]

S. LallJ. Marsden and S. Glavaski, A subspace approach to balanced truncation for model reduction of nonlinear control systems, nt. J. on Robust and Nonl. Contr., 12 (2002), 519-535.  doi: 10.1002/rnc.657.  Google Scholar

[28]

D. Liberzon and R. W. Brockett, Nonlinear feedback systems perturbed by noise: Steady-state probability distributions and optimal control, IEEE T. Automat. Control, 45 (2000), 1116-1130.  doi: 10.1109/9.863596.  Google Scholar

[29]

B. Moore, Principal component analysis in linear systems: Controllability, observability, and model reduction, IEEE T. Automat. Control, 26 (1981), 17-32.  doi: 10.1109/TAC.1981.1102568.  Google Scholar

[30]

A. J. Newman and P. S. Krishnaprasad, Computing balanced realizations for nonlinear systems, Proc. of the Math. Theory of Networks and Systems (MTNS), (2000). Google Scholar

[31]

H. Risken, The Fokker-Planck Equation, Springer, 1984. doi: 10.1007/978-3-642-96807-5.  Google Scholar

[32]

L. RosascoM. Belkin and E. De BVito, On learning with integral operators, J. Mach. Learn. Res., 11 (2010), 905-934.   Google Scholar

[33]

C. W. Rowley, Model reduction for fluids using balanced proper orthogonal decomposition, Int. J. Bifurcat. Chaos, 11 (2010), 905-934.  doi: 10.1142/S0218127405012429.  Google Scholar

[34]

J. M. A Scherpen, Balancing for nonlinear systems, Systems & Control Letters, 21 (1993), 143-153.  doi: 10.1016/0167-6911(93)90117-O.  Google Scholar

[35]

B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines,Regularization, Optimization, and Beyond, MIT Press, 2001. Google Scholar

[36]

S. Smale and D. X. Zhou, Learning theory estimates via integral operators and their approximations, Constr. Approx., 26 (2007), 153-172.  doi: 10.1007/s00365-006-0659-y.  Google Scholar

[37]

G. Wahba, Spline Models for Observational Data, SIAM CBMS-NSF Regional Conference Series in Applied Mathematics, Philadelphia, PA, 1990. doi: 10.1137/1.9781611970128.  Google Scholar

[38]

H. Wendland, Scattered Data Approximation, Cambridge Monogr. Appl. Comput. Math., Cambridge University Press, Cambridge, UK, 2005.  Google Scholar

[39]

M. Zakai, A Lyapunov criterion for the existence of stationary probability distributions for systems perturbed by noise, SIAM J. Control, 1 (1969), 390-397.  doi: 10.1137/0307028.  Google Scholar

show all references

References:
[1]

N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc., 68 (1950), 337-404.  doi: 10.1090/S0002-9947-1950-0051437-7.  Google Scholar

[2]

G. BiauB. Cadre and B. Pelletier, Exact rates in density support estimation, J. Multivariate Anal., 99 (2008), 2185-2207.  doi: 10.1016/j.jmva.2008.02.021.  Google Scholar

[3]

V. I. Bogachev, Gaussian Measures, American Mathematical Society, 1998.  Google Scholar

[4]

J. Bouvrie and B. Hamzi, Balanced reduction of nonlinear control systems in reproducing kernel Hilbert space, in Proc. 48th Annual Allerton Conference on Communication, Control, and Computing, (2010), 294-301, http://arxiv.org/abs/1011.2952. doi: 10.1109/ALLERTON.2010.5706920.  Google Scholar

[5]

J. Bouvrie and B. Hamzi, Kernel methods for the approximation of nonlinear systems, in SIAM Journal on Control and Optimization, (2017), to appear, https://arxiv.org/abs/1108.2903. Google Scholar

[6]

J. Bouvrie and B. Hamzi, Empirical estimators for stochastically forced nonlinear systems: Observability, controllability and the invariant measure, in Proc. American Control Conference (ACC), 2012, (2012). doi: 10.1109/ACC.2012.6315175.  Google Scholar

[7]

R. Brockett, Stochastic Control, Lecture Notes, Harvard University Press, 2009. Google Scholar

[8]

R. L. Butchart, An explicit solution to the Fokker-Planck equation for an ordinary differential equation, Int. J. Control, 1 (1965), 201-208.  doi: 10.1080/00207176508905472.  Google Scholar

[9]

A. Caponnetto and E. De Vito, Optimal rates for the regularized least-squares algorithm, Found. Comput. Math., 7 (2007), 331-368.  doi: 10.1007/s10208-006-0196-8.  Google Scholar

[10]

F. Cucker and S. Smale, On the mathematical foundations of learning, Bull. AMS, 39 (2002), 1-49.  doi: 10.1090/S0273-0979-01-00923-5.  Google Scholar

[11]

G. Da Prato, An Introduction to Infinite Dimensional Analysis, Springer, 2006. doi: 10.1007/3-540-29021-4.  Google Scholar

[12]

G. Da Prato and J. Zabczyk, Stochastic Equations in Infinite Dimensions, Cambridge University Press, 1992. doi: 10.1017/CBO9780511666223.  Google Scholar

[13]

E. De Vito, L. Rosasco and A. Toigo, Spectral Regularization for Support Estimation, in J. Shawe-Taylor et al., eds., Advances in Neural Information Processing Systems (NIPS), 24, Vancouver, Curran Associates, Inc., 2010. Google Scholar

[14]

G. E. Dullerud and F. Paganini, A Course in Robust Control Theory: A Convex Approach, Springer, 2000. doi: 10.1007/978-1-4757-3290-0.  Google Scholar

[15]

G. FroylandK. JuddA. I. MeesK. Murao and D. Watson, Constructing invariant measures from data, Int. J. Bifurcat. Chaos, 5 (1995), 1181-1192.   Google Scholar

[16]

G. Froyland, Extracting dynamical behaviour via Markov models, In Alistair Mees, ed., Nonlinear Dynamics and Statistics: Proceedings, Newton Institute, Cambridge, (2001), 281–321, Birkhauser.  Google Scholar

[17]

K. Fujimoto and D. Tsubakino, Computation of nonlinear balanced realization and model reduction based on Taylor series expansion, Systems and Control Letters, 57 (2008), 283-289.  doi: 10.1016/j.sysconle.2007.08.015.  Google Scholar

[18]

A. T. Fuller, Analysis of nonlinear stochastic systems by means of the Fokker-Planck equation, Int. J. Control, 9 (1969), 603-655.  doi: 10.1007/3-540-29021-4.  Google Scholar

[19]

P. Giesl and H. Wendland, Meshless collocation: Error estimates with application to dynamical systems, SIAM J. Num. Anal., 45 (2007), 1723-1749.  doi: 10.1137/060658813.  Google Scholar

[20]

P. Giesl, Construction of Global Lyapunov Functions Using Radial Basis Functions, Springer, 2007.  Google Scholar

[21]

W. S. Gray and E. I. Verriest, Algebraically defined gramians for nonlinear systems, Proc. of the 45th IEEE CDC , (2006). doi: 10.1109/CDC.2006.376840.  Google Scholar

[22]

J. GuinezR. Quintero and A. D. Rueda, Calculating steady states for a Fokker-Planck equation, Acta Math. Hungar., 91 (2001), 311-323.  doi: 10.1023/A:1010615818034.  Google Scholar

[23]

C. Hartmann and C. Schuette, Balancing of partially-observed stochastic differential equations, Proc. of the 47th IEEE CDC, (2008), 4867-4872.  doi: 10.1007/3-540-29021-4.  Google Scholar

[24]

D. KilminsterD. Allingham and A. Mees, Estimating invariant probability densities for dynamical systems: Nonparametric approach to time series analysis, Ann. Ⅰ. Stat. Math., 39 (2002), 1-49.  doi: 10.1023/A:1016134209348.  Google Scholar

[25]

A. J. Krener, The Important State Coordinates of a Nonlinear System, In Advances in control theory and applications, C. Bonivento, A. Isidori, L. Marconi, C. Rossi, editors, 353 (2007), 161-170, Springer. doi: 10.1007/978-3-540-70701-1_8.  Google Scholar

[26]

A. J. Krener, Reduced order modeling of nonlinear control systems, In Analysis and Design of Nonlinear Control Systems, A. Astolfi and L. Marconi, editors, (2008), 41-62, Springer. doi: 10.1007/978-3-540-74358-3_4.  Google Scholar

[27]

S. LallJ. Marsden and S. Glavaski, A subspace approach to balanced truncation for model reduction of nonlinear control systems, nt. J. on Robust and Nonl. Contr., 12 (2002), 519-535.  doi: 10.1002/rnc.657.  Google Scholar

[28]

D. Liberzon and R. W. Brockett, Nonlinear feedback systems perturbed by noise: Steady-state probability distributions and optimal control, IEEE T. Automat. Control, 45 (2000), 1116-1130.  doi: 10.1109/9.863596.  Google Scholar

[29]

B. Moore, Principal component analysis in linear systems: Controllability, observability, and model reduction, IEEE T. Automat. Control, 26 (1981), 17-32.  doi: 10.1109/TAC.1981.1102568.  Google Scholar

[30]

A. J. Newman and P. S. Krishnaprasad, Computing balanced realizations for nonlinear systems, Proc. of the Math. Theory of Networks and Systems (MTNS), (2000). Google Scholar

[31]

H. Risken, The Fokker-Planck Equation, Springer, 1984. doi: 10.1007/978-3-642-96807-5.  Google Scholar

[32]

L. RosascoM. Belkin and E. De BVito, On learning with integral operators, J. Mach. Learn. Res., 11 (2010), 905-934.   Google Scholar

[33]

C. W. Rowley, Model reduction for fluids using balanced proper orthogonal decomposition, Int. J. Bifurcat. Chaos, 11 (2010), 905-934.  doi: 10.1142/S0218127405012429.  Google Scholar

[34]

J. M. A Scherpen, Balancing for nonlinear systems, Systems & Control Letters, 21 (1993), 143-153.  doi: 10.1016/0167-6911(93)90117-O.  Google Scholar

[35]

B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines,Regularization, Optimization, and Beyond, MIT Press, 2001. Google Scholar

[36]

S. Smale and D. X. Zhou, Learning theory estimates via integral operators and their approximations, Constr. Approx., 26 (2007), 153-172.  doi: 10.1007/s00365-006-0659-y.  Google Scholar

[37]

G. Wahba, Spline Models for Observational Data, SIAM CBMS-NSF Regional Conference Series in Applied Mathematics, Philadelphia, PA, 1990. doi: 10.1137/1.9781611970128.  Google Scholar

[38]

H. Wendland, Scattered Data Approximation, Cambridge Monogr. Appl. Comput. Math., Cambridge University Press, Cambridge, UK, 2005.  Google Scholar

[39]

M. Zakai, A Lyapunov criterion for the existence of stationary probability distributions for systems perturbed by noise, SIAM J. Control, 1 (1969), 390-397.  doi: 10.1137/0307028.  Google Scholar

Figure 1.  Comparison between the exact (red), the kernel-based (green) and the empirical (black) steady-state distribution
[1]

Xuefei He, Kun Wang, Liwei Xu. Efficient finite difference methods for the nonlinear Helmholtz equation in Kerr medium. Electronic Research Archive, 2020, 28 (4) : 1503-1528. doi: 10.3934/era.2020079

[2]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444

[3]

Bo Chen, Youde Wang. Global weak solutions for Landau-Lifshitz flows and heat flows associated to micromagnetic energy functional. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020268

[4]

Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115

[5]

Reza Lotfi, Zahra Yadegari, Seyed Hossein Hosseini, Amir Hossein Khameneh, Erfan Babaee Tirkolaee, Gerhard-Wilhelm Weber. A robust time-cost-quality-energy-environment trade-off with resource-constrained in project management: A case study for a bridge construction project. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020158

[6]

Soniya Singh, Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of second order impulsive systems with state-dependent delay in Banach spaces. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020103

[7]

Thierry Cazenave, Ivan Naumkin. Local smooth solutions of the nonlinear Klein-gordon equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020448

[8]

Maoding Zhen, Binlin Zhang, Vicenţiu D. Rădulescu. Normalized solutions for nonlinear coupled fractional systems: Low and high perturbations in the attractive case. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020379

[9]

Zedong Yang, Guotao Wang, Ravi P. Agarwal, Haiyong Xu. Existence and nonexistence of entire positive radial solutions for a class of Schrödinger elliptic systems involving a nonlinear operator. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020436

[10]

Xin Guo, Lexin Li, Qiang Wu. Modeling interactive components by coordinate kernel polynomial models. Mathematical Foundations of Computing, 2020, 3 (4) : 263-277. doi: 10.3934/mfc.2020010

[11]

Kihoon Seong. Low regularity a priori estimates for the fourth order cubic nonlinear Schrödinger equation. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5437-5473. doi: 10.3934/cpaa.2020247

[12]

José Luis López. A quantum approach to Keller-Segel dynamics via a dissipative nonlinear Schrödinger equation. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020376

[13]

Claudianor O. Alves, Rodrigo C. M. Nemer, Sergio H. Monari Soares. The use of the Morse theory to estimate the number of nontrivial solutions of a nonlinear Schrödinger equation with a magnetic field. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020276

[14]

Ying Lin, Qi Ye. Support vector machine classifiers by non-Euclidean margins. Mathematical Foundations of Computing, 2020, 3 (4) : 279-300. doi: 10.3934/mfc.2020018

[15]

Justin Holmer, Chang Liu. Blow-up for the 1D nonlinear Schrödinger equation with point nonlinearity II: Supercritical blow-up profiles. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020264

[16]

Bahaaeldin Abdalla, Thabet Abdeljawad. Oscillation criteria for kernel function dependent fractional dynamic equations. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020443

[17]

Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of a Sobolev type impulsive functional evolution system in Banach spaces. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020049

[18]

Xin Guo, Lei Shi. Preface of the special issue on analysis in data science: Methods and applications. Mathematical Foundations of Computing, 2020, 3 (4) : i-ii. doi: 10.3934/mfc.2020026

[19]

Zhilei Liang, Jiangyu Shuai. Existence of strong solution for the Cauchy problem of fully compressible Navier-Stokes equations in two dimensions. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020348

[20]

Thabet Abdeljawad, Mohammad Esmael Samei. Applying quantum calculus for the existence of solution of $ q $-integro-differential equations with three criteria. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020440

 Impact Factor: 

Metrics

  • PDF downloads (182)
  • HTML views (2130)
  • Cited by (1)

Other articles
by authors

[Back to Top]