June  2012, 17(4): 1333-1363. doi: 10.3934/dcdsb.2012.17.1333

Fundamental limitations of Ad hoc linear and quadratic multi-level regression models for physical systems

1. 

Department of Mathematics and Center for Atmosphere and Ocean Science, Courant Institute for Mathematical Sciences, New York University, New York, NY 10012-1110, United States

2. 

Courant Institute for Mathematical Sciences, New York University, New York, NY 10012-1110, United States

Received  November 2010 Revised  January 2012 Published  February 2012

A central issue in contemporary applied mathematics is the development of simpler dynamical models for a reduced subset of variables in complex high dimensional dynamical systems with many spatio-temporal scales. Recently, ad hoc quadratic multi-level regression models have been proposed to provide suitable reduced nonlinear models directly from data. The main results developed here are rigorous theorems demonstrating the non-physical finite time blow-up and large time instability in statistical solutions of general scalar multi-level quadratic regression models with corresponding unphysical features of the invariant measure. Surprising intrinsic model errors due to discrete sampling errors are also shown to occur rigorously even for linear multi-level regression dynamic models. all of these theoretical results are corroborated by numerical experiments with simple models. Single level nonlinear regression strategies with physical cubic damping are shown to have significant skill on the same test problems.
Citation: Andrew J. Majda, Yuan Yuan. Fundamental limitations of Ad hoc linear and quadratic multi-level regression models for physical systems. Discrete and Continuous Dynamical Systems - B, 2012, 17 (4) : 1333-1363. doi: 10.3934/dcdsb.2012.17.1333
References:
[1]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, Models for stochastic climate prediction, PNAS, 96 (1999), 14687-14691. doi: 10.1073/pnas.96.26.14687.

[2]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, A mathematical framework for stochastic climate models, Commun. Pure Appl. Math., 54 (2001), 891-974. doi: 10.1002/cpa.1014.

[3]

A. J. Majda, C. Franzke and B. Khouider, An applied mathematics perspective on stochastic modelling for climate, Phil. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 366 (2008), 2429-2455.

[4]

A. J. Majda, R. Abramov and M. Grote, "Information Theory and Stochastics for Multiscale Nonlinear Systems," CRM Monograph Series, 25, American Mathematical Society, Providence, RI, 2005.

[5]

I. Horenko, Finite element approach to clustering of multidimensional time series, SIAM J. Sci. Comput., 32 (2010), 62-83. doi: 10.1137/080715962.

[6]

I. Horenko, On the identification of nonstationary factor models and their application to atmospheric data analysis, J. Atmos. Sci., 67 (2010), 1559-1574. doi: 10.1175/2010JAS3271.1.

[7]

D. Crommelin and E. Vanden-Eijnden, Subgrid-scale parameterization with conditional Markov chains, J. Atmos. Sci., 65 (2008), 2661-2675. doi: 10.1175/2008JAS2566.1.

[8]

A. J. Majda, C. Franzke and D. Crommelin, Normal forms for reduced stochastic climate models, PNAS, 106 (2009), 3649-3653. doi: 10.1073/pnas.0900173106.

[9]

A. J. Majda, C. Franzke, A. Fischer and D. Crommelin, Distinct metastable atmospheric regimes despite nearly Gaussian statistics: A paradigm model, PNAS, 103 (2006), 8309-8314. doi: 10.1073/pnas.0602641103.

[10]

C. Franzke, A. J. Majda and E. Vanden-Eijnden, Low-order stochastic mode reduction for a realistic barotropic model climate, J. Atmos. Sci., 62 (2005), 1722-1745. doi: 10.1175/JAS3438.1.

[11]

T. Delsole, Stochastic models of quasigeostrophic turbulence, Surv. Geophys., 25 (2004), 107-149.

[12]

A. J. Majda, J. Harlim and B. Gershgorin, Mathematical strategies for filtering turbulent dynamical systems, Discrete and Continuous Dynamical Systems, 27 (2010), 441-486. doi: 10.3934/dcds.2010.27.441.

[13]

A. J. Majda, B. Gershgorin and Y. Yuan, Low frequency response and fluctuation-dissipation Theorems: Theory and practice, J. Atmos. Sci., 67 (2010), 1186-1201. doi: 10.1175/2009JAS3264.1.

[14]

A. J. Majda, R. Abramov and B. Gershgorin, High skill in low-frequency climate response through fluctuation dissipation theorems despite structural instability, PNAS, 107 (2010), 581-586. doi: 10.1073/pnas.0912997107.

[15]

S. Kravtsov, D. Kondrashov and M. Ghil, Multilevel regression modeling of nonlinear processes: Derivation and applications to climatic variability, J. Climate, 18 (2005), 4404-4424. doi: 10.1175/JCLI3544.1.

[16]

D. Kondrashov, S. Kravtsov, A. W. Robertson and M. Ghil, A hierarchy of data-based ENSO models, J. Climate, 18 (2005), 4425-4444. doi: 10.1175/JCLI3567.1.

[17]

D. Kondrashov, S. Kravtsov and M. Ghil, Empirical mode reduction in a model of extratropical low-frequency variability, J. Atmos. Sci, 63 (2006), 1859-1877. doi: 10.1175/JAS3719.1.

[18]

G. A. Pavliotis and A. M. Stuart, Parameter estimation for multiscale diffusions, J. Stat. Phys., 127 (2007), 741-781. doi: 10.1007/s10955-007-9300-6.

[19]

R. Azencott, A. Beri and I. Timofeyev, Adaptive sub-sampling for parametric estimation of Gaussian diffusions, J. Stat. Phys., 139 (2010), 1066-1089. doi: 10.1007/s10955-010-9975-y.

[20]

R. Azencott, A. Beri and I. Timofeyev, Sub-sampling and parametric estimation for multi-scale dynamics, submitted, 2010.

[21]

Y. Yuan and A. J. Majda, Invariant measures and asymptotic gaussian bounds for normal forms of stochastic climate model, Chinese Annals of Mathematics, Series B, 32 (2011), 343-368. doi: 10.1007/s11401-011-0647-2.

[22]

A. J. Majda and B. Gershgorin, Quantifying uncertainty in climate change science through empirical information theory, PNAS, 107 (2010), 14958-14963. doi: 10.1073/pnas.1007009107.

[23]

C. Franzke, I. Horenko, A. J. Majda and R. Klein, Systematic metastable atmospheric regime identification in a AGCM, J. Atmos. Sci., 66 (2009), 1997-2012. doi: 10.1175/2009JAS2939.1.

[24]

J. Berner and G. Branstator, Linear and nonlinear signatures in planetary wave dynamics of an AGCM: Probability density functions, J. Atmos. Sci., 64 (2007), 117-136. doi: 10.1175/JAS3822.1.

[25]

R. Z. Has'minskiĭ, A limit theorem for solutions of differential equations with a random right-hand part, Teor. Verojatnost. i Primenen, 11 (1966), 444-462.

[26]

T. G. Kurtz, A limit theorem for perturbed operator semigroups with applications to random evolutions, J. Functional Analysis, 12 (1973), 55-67. doi: 10.1016/0022-1236(73)90089-X.

show all references

References:
[1]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, Models for stochastic climate prediction, PNAS, 96 (1999), 14687-14691. doi: 10.1073/pnas.96.26.14687.

[2]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, A mathematical framework for stochastic climate models, Commun. Pure Appl. Math., 54 (2001), 891-974. doi: 10.1002/cpa.1014.

[3]

A. J. Majda, C. Franzke and B. Khouider, An applied mathematics perspective on stochastic modelling for climate, Phil. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 366 (2008), 2429-2455.

[4]

A. J. Majda, R. Abramov and M. Grote, "Information Theory and Stochastics for Multiscale Nonlinear Systems," CRM Monograph Series, 25, American Mathematical Society, Providence, RI, 2005.

[5]

I. Horenko, Finite element approach to clustering of multidimensional time series, SIAM J. Sci. Comput., 32 (2010), 62-83. doi: 10.1137/080715962.

[6]

I. Horenko, On the identification of nonstationary factor models and their application to atmospheric data analysis, J. Atmos. Sci., 67 (2010), 1559-1574. doi: 10.1175/2010JAS3271.1.

[7]

D. Crommelin and E. Vanden-Eijnden, Subgrid-scale parameterization with conditional Markov chains, J. Atmos. Sci., 65 (2008), 2661-2675. doi: 10.1175/2008JAS2566.1.

[8]

A. J. Majda, C. Franzke and D. Crommelin, Normal forms for reduced stochastic climate models, PNAS, 106 (2009), 3649-3653. doi: 10.1073/pnas.0900173106.

[9]

A. J. Majda, C. Franzke, A. Fischer and D. Crommelin, Distinct metastable atmospheric regimes despite nearly Gaussian statistics: A paradigm model, PNAS, 103 (2006), 8309-8314. doi: 10.1073/pnas.0602641103.

[10]

C. Franzke, A. J. Majda and E. Vanden-Eijnden, Low-order stochastic mode reduction for a realistic barotropic model climate, J. Atmos. Sci., 62 (2005), 1722-1745. doi: 10.1175/JAS3438.1.

[11]

T. Delsole, Stochastic models of quasigeostrophic turbulence, Surv. Geophys., 25 (2004), 107-149.

[12]

A. J. Majda, J. Harlim and B. Gershgorin, Mathematical strategies for filtering turbulent dynamical systems, Discrete and Continuous Dynamical Systems, 27 (2010), 441-486. doi: 10.3934/dcds.2010.27.441.

[13]

A. J. Majda, B. Gershgorin and Y. Yuan, Low frequency response and fluctuation-dissipation Theorems: Theory and practice, J. Atmos. Sci., 67 (2010), 1186-1201. doi: 10.1175/2009JAS3264.1.

[14]

A. J. Majda, R. Abramov and B. Gershgorin, High skill in low-frequency climate response through fluctuation dissipation theorems despite structural instability, PNAS, 107 (2010), 581-586. doi: 10.1073/pnas.0912997107.

[15]

S. Kravtsov, D. Kondrashov and M. Ghil, Multilevel regression modeling of nonlinear processes: Derivation and applications to climatic variability, J. Climate, 18 (2005), 4404-4424. doi: 10.1175/JCLI3544.1.

[16]

D. Kondrashov, S. Kravtsov, A. W. Robertson and M. Ghil, A hierarchy of data-based ENSO models, J. Climate, 18 (2005), 4425-4444. doi: 10.1175/JCLI3567.1.

[17]

D. Kondrashov, S. Kravtsov and M. Ghil, Empirical mode reduction in a model of extratropical low-frequency variability, J. Atmos. Sci, 63 (2006), 1859-1877. doi: 10.1175/JAS3719.1.

[18]

G. A. Pavliotis and A. M. Stuart, Parameter estimation for multiscale diffusions, J. Stat. Phys., 127 (2007), 741-781. doi: 10.1007/s10955-007-9300-6.

[19]

R. Azencott, A. Beri and I. Timofeyev, Adaptive sub-sampling for parametric estimation of Gaussian diffusions, J. Stat. Phys., 139 (2010), 1066-1089. doi: 10.1007/s10955-010-9975-y.

[20]

R. Azencott, A. Beri and I. Timofeyev, Sub-sampling and parametric estimation for multi-scale dynamics, submitted, 2010.

[21]

Y. Yuan and A. J. Majda, Invariant measures and asymptotic gaussian bounds for normal forms of stochastic climate model, Chinese Annals of Mathematics, Series B, 32 (2011), 343-368. doi: 10.1007/s11401-011-0647-2.

[22]

A. J. Majda and B. Gershgorin, Quantifying uncertainty in climate change science through empirical information theory, PNAS, 107 (2010), 14958-14963. doi: 10.1073/pnas.1007009107.

[23]

C. Franzke, I. Horenko, A. J. Majda and R. Klein, Systematic metastable atmospheric regime identification in a AGCM, J. Atmos. Sci., 66 (2009), 1997-2012. doi: 10.1175/2009JAS2939.1.

[24]

J. Berner and G. Branstator, Linear and nonlinear signatures in planetary wave dynamics of an AGCM: Probability density functions, J. Atmos. Sci., 64 (2007), 117-136. doi: 10.1175/JAS3822.1.

[25]

R. Z. Has'minskiĭ, A limit theorem for solutions of differential equations with a random right-hand part, Teor. Verojatnost. i Primenen, 11 (1966), 444-462.

[26]

T. G. Kurtz, A limit theorem for perturbed operator semigroups with applications to random evolutions, J. Functional Analysis, 12 (1973), 55-67. doi: 10.1016/0022-1236(73)90089-X.

[1]

Tayeb Hadj Kaddour, Michael Reissig. Blow-up results for effectively damped wave models with nonlinear memory. Communications on Pure and Applied Analysis, 2021, 20 (7&8) : 2687-2707. doi: 10.3934/cpaa.2020239

[2]

Alex H. Ardila, Mykael Cardoso. Blow-up solutions and strong instability of ground states for the inhomogeneous nonlinear Schrödinger equation. Communications on Pure and Applied Analysis, 2021, 20 (1) : 101-119. doi: 10.3934/cpaa.2020259

[3]

Hayato Miyazaki. Strong blow-up instability for standing wave solutions to the system of the quadratic nonlinear Klein-Gordon equations. Discrete and Continuous Dynamical Systems, 2021, 41 (5) : 2411-2445. doi: 10.3934/dcds.2020370

[4]

María J. Cáceres, Ricarda Schneider. Blow-up, steady states and long time behaviour of excitatory-inhibitory nonlinear neuron models. Kinetic and Related Models, 2017, 10 (3) : 587-612. doi: 10.3934/krm.2017024

[5]

Quang-Minh Tran, Hong-Danh Pham. Global existence and blow-up results for a nonlinear model for a dynamic suspension bridge. Discrete and Continuous Dynamical Systems - S, 2021, 14 (12) : 4521-4550. doi: 10.3934/dcdss.2021135

[6]

Jorge A. Esquivel-Avila. Blow-up in damped abstract nonlinear equations. Electronic Research Archive, 2020, 28 (1) : 347-367. doi: 10.3934/era.2020020

[7]

Victor A. Galaktionov, Juan-Luis Vázquez. The problem Of blow-up in nonlinear parabolic equations. Discrete and Continuous Dynamical Systems, 2002, 8 (2) : 399-433. doi: 10.3934/dcds.2002.8.399

[8]

José R. Quintero, Juan C. Cordero. Instability of the standing waves for a Benney-Roskes/Zakharov-Rubenchik system and blow-up for the Zakharov equations. Discrete and Continuous Dynamical Systems - B, 2020, 25 (4) : 1213-1240. doi: 10.3934/dcdsb.2019217

[9]

Mahdi Roozbeh, Saman Babaie–Kafaki, Zohre Aminifard. Two penalized mixed–integer nonlinear programming approaches to tackle multicollinearity and outliers effects in linear regression models. Journal of Industrial and Management Optimization, 2021, 17 (6) : 3475-3491. doi: 10.3934/jimo.2020128

[10]

Evgeny Galakhov, Olga Salieva. Blow-up for nonlinear inequalities with gradient terms and singularities on unbounded sets. Conference Publications, 2015, 2015 (special) : 489-494. doi: 10.3934/proc.2015.0489

[11]

Jong-Shenq Guo. Blow-up behavior for a quasilinear parabolic equation with nonlinear boundary condition. Discrete and Continuous Dynamical Systems, 2007, 18 (1) : 71-84. doi: 10.3934/dcds.2007.18.71

[12]

Pavol Quittner, Philippe Souplet. Blow-up rate of solutions of parabolic poblems with nonlinear boundary conditions. Discrete and Continuous Dynamical Systems - S, 2012, 5 (3) : 671-681. doi: 10.3934/dcdss.2012.5.671

[13]

Satyanad Kichenassamy. Control of blow-up singularities for nonlinear wave equations. Evolution Equations and Control Theory, 2013, 2 (4) : 669-677. doi: 10.3934/eect.2013.2.669

[14]

Monica Marras, Stella Vernier Piro. Bounds for blow-up time in nonlinear parabolic systems. Conference Publications, 2011, 2011 (Special) : 1025-1031. doi: 10.3934/proc.2011.2011.1025

[15]

Dapeng Du, Yifei Wu, Kaijun Zhang. On blow-up criterion for the nonlinear Schrödinger equation. Discrete and Continuous Dynamical Systems, 2016, 36 (7) : 3639-3650. doi: 10.3934/dcds.2016.36.3639

[16]

Walter A. Strauss, Kimitoshi Tsutaya. Existence and blow up of small amplitude nonlinear waves with a negative potential. Discrete and Continuous Dynamical Systems, 1997, 3 (2) : 175-188. doi: 10.3934/dcds.1997.3.175

[17]

Laurent Di Menza, Olivier Goubet. Stabilizing blow up solutions to nonlinear schrÖdinger equations. Communications on Pure and Applied Analysis, 2017, 16 (3) : 1059-1082. doi: 10.3934/cpaa.2017051

[18]

Antonio Vitolo, Maria E. Amendola, Giulio Galise. On the uniqueness of blow-up solutions of fully nonlinear elliptic equations. Conference Publications, 2013, 2013 (special) : 771-780. doi: 10.3934/proc.2013.2013.771

[19]

Françoise Demengel, O. Goubet. Existence of boundary blow up solutions for singular or degenerate fully nonlinear equations. Communications on Pure and Applied Analysis, 2013, 12 (2) : 621-645. doi: 10.3934/cpaa.2013.12.621

[20]

Filippo Gazzola, Paschalis Karageorgis. Refined blow-up results for nonlinear fourth order differential equations. Communications on Pure and Applied Analysis, 2015, 14 (2) : 677-693. doi: 10.3934/cpaa.2015.14.677

2020 Impact Factor: 1.327

Metrics

  • PDF downloads (124)
  • HTML views (0)
  • Cited by (18)

Other articles
by authors

[Back to Top]