June  2012, 17(4): 1333-1363. doi: 10.3934/dcdsb.2012.17.1333

Fundamental limitations of Ad hoc linear and quadratic multi-level regression models for physical systems

1. 

Department of Mathematics and Center for Atmosphere and Ocean Science, Courant Institute for Mathematical Sciences, New York University, New York, NY 10012-1110, United States

2. 

Courant Institute for Mathematical Sciences, New York University, New York, NY 10012-1110, United States

Received  November 2010 Revised  January 2012 Published  February 2012

A central issue in contemporary applied mathematics is the development of simpler dynamical models for a reduced subset of variables in complex high dimensional dynamical systems with many spatio-temporal scales. Recently, ad hoc quadratic multi-level regression models have been proposed to provide suitable reduced nonlinear models directly from data. The main results developed here are rigorous theorems demonstrating the non-physical finite time blow-up and large time instability in statistical solutions of general scalar multi-level quadratic regression models with corresponding unphysical features of the invariant measure. Surprising intrinsic model errors due to discrete sampling errors are also shown to occur rigorously even for linear multi-level regression dynamic models. all of these theoretical results are corroborated by numerical experiments with simple models. Single level nonlinear regression strategies with physical cubic damping are shown to have significant skill on the same test problems.
Citation: Andrew J. Majda, Yuan Yuan. Fundamental limitations of Ad hoc linear and quadratic multi-level regression models for physical systems. Discrete & Continuous Dynamical Systems - B, 2012, 17 (4) : 1333-1363. doi: 10.3934/dcdsb.2012.17.1333
References:
[1]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, Models for stochastic climate prediction,, PNAS, 96 (1999), 14687.  doi: 10.1073/pnas.96.26.14687.  Google Scholar

[2]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, A mathematical framework for stochastic climate models,, Commun. Pure Appl. Math., 54 (2001), 891.  doi: 10.1002/cpa.1014.  Google Scholar

[3]

A. J. Majda, C. Franzke and B. Khouider, An applied mathematics perspective on stochastic modelling for climate,, Phil. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 366 (2008), 2429.   Google Scholar

[4]

A. J. Majda, R. Abramov and M. Grote, "Information Theory and Stochastics for Multiscale Nonlinear Systems,", CRM Monograph Series, 25 (2005).   Google Scholar

[5]

I. Horenko, Finite element approach to clustering of multidimensional time series,, SIAM J. Sci. Comput., 32 (2010), 62.  doi: 10.1137/080715962.  Google Scholar

[6]

I. Horenko, On the identification of nonstationary factor models and their application to atmospheric data analysis,, J. Atmos. Sci., 67 (2010), 1559.  doi: 10.1175/2010JAS3271.1.  Google Scholar

[7]

D. Crommelin and E. Vanden-Eijnden, Subgrid-scale parameterization with conditional Markov chains,, J. Atmos. Sci., 65 (2008), 2661.  doi: 10.1175/2008JAS2566.1.  Google Scholar

[8]

A. J. Majda, C. Franzke and D. Crommelin, Normal forms for reduced stochastic climate models,, PNAS, 106 (2009), 3649.  doi: 10.1073/pnas.0900173106.  Google Scholar

[9]

A. J. Majda, C. Franzke, A. Fischer and D. Crommelin, Distinct metastable atmospheric regimes despite nearly Gaussian statistics: A paradigm model,, PNAS, 103 (2006), 8309.  doi: 10.1073/pnas.0602641103.  Google Scholar

[10]

C. Franzke, A. J. Majda and E. Vanden-Eijnden, Low-order stochastic mode reduction for a realistic barotropic model climate,, J. Atmos. Sci., 62 (2005), 1722.  doi: 10.1175/JAS3438.1.  Google Scholar

[11]

T. Delsole, Stochastic models of quasigeostrophic turbulence,, Surv. Geophys., 25 (2004), 107.   Google Scholar

[12]

A. J. Majda, J. Harlim and B. Gershgorin, Mathematical strategies for filtering turbulent dynamical systems,, Discrete and Continuous Dynamical Systems, 27 (2010), 441.  doi: 10.3934/dcds.2010.27.441.  Google Scholar

[13]

A. J. Majda, B. Gershgorin and Y. Yuan, Low frequency response and fluctuation-dissipation Theorems: Theory and practice,, J. Atmos. Sci., 67 (2010), 1186.  doi: 10.1175/2009JAS3264.1.  Google Scholar

[14]

A. J. Majda, R. Abramov and B. Gershgorin, High skill in low-frequency climate response through fluctuation dissipation theorems despite structural instability,, PNAS, 107 (2010), 581.  doi: 10.1073/pnas.0912997107.  Google Scholar

[15]

S. Kravtsov, D. Kondrashov and M. Ghil, Multilevel regression modeling of nonlinear processes: Derivation and applications to climatic variability,, J. Climate, 18 (2005), 4404.  doi: 10.1175/JCLI3544.1.  Google Scholar

[16]

D. Kondrashov, S. Kravtsov, A. W. Robertson and M. Ghil, A hierarchy of data-based ENSO models,, J. Climate, 18 (2005), 4425.  doi: 10.1175/JCLI3567.1.  Google Scholar

[17]

D. Kondrashov, S. Kravtsov and M. Ghil, Empirical mode reduction in a model of extratropical low-frequency variability,, J. Atmos. Sci, 63 (2006), 1859.  doi: 10.1175/JAS3719.1.  Google Scholar

[18]

G. A. Pavliotis and A. M. Stuart, Parameter estimation for multiscale diffusions,, J. Stat. Phys., 127 (2007), 741.  doi: 10.1007/s10955-007-9300-6.  Google Scholar

[19]

R. Azencott, A. Beri and I. Timofeyev, Adaptive sub-sampling for parametric estimation of Gaussian diffusions,, J. Stat. Phys., 139 (2010), 1066.  doi: 10.1007/s10955-010-9975-y.  Google Scholar

[20]

R. Azencott, A. Beri and I. Timofeyev, Sub-sampling and parametric estimation for multi-scale dynamics,, submitted, (2010).   Google Scholar

[21]

Y. Yuan and A. J. Majda, Invariant measures and asymptotic gaussian bounds for normal forms of stochastic climate model,, Chinese Annals of Mathematics, 32 (2011), 343.  doi: 10.1007/s11401-011-0647-2.  Google Scholar

[22]

A. J. Majda and B. Gershgorin, Quantifying uncertainty in climate change science through empirical information theory,, PNAS, 107 (2010), 14958.  doi: 10.1073/pnas.1007009107.  Google Scholar

[23]

C. Franzke, I. Horenko, A. J. Majda and R. Klein, Systematic metastable atmospheric regime identification in a AGCM,, J. Atmos. Sci., 66 (2009), 1997.  doi: 10.1175/2009JAS2939.1.  Google Scholar

[24]

J. Berner and G. Branstator, Linear and nonlinear signatures in planetary wave dynamics of an AGCM: Probability density functions,, J. Atmos. Sci., 64 (2007), 117.  doi: 10.1175/JAS3822.1.  Google Scholar

[25]

R. Z. Has'minskiĭ, A limit theorem for solutions of differential equations with a random right-hand part,, Teor. Verojatnost. i Primenen, 11 (1966), 444.   Google Scholar

[26]

T. G. Kurtz, A limit theorem for perturbed operator semigroups with applications to random evolutions,, J. Functional Analysis, 12 (1973), 55.  doi: 10.1016/0022-1236(73)90089-X.  Google Scholar

show all references

References:
[1]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, Models for stochastic climate prediction,, PNAS, 96 (1999), 14687.  doi: 10.1073/pnas.96.26.14687.  Google Scholar

[2]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, A mathematical framework for stochastic climate models,, Commun. Pure Appl. Math., 54 (2001), 891.  doi: 10.1002/cpa.1014.  Google Scholar

[3]

A. J. Majda, C. Franzke and B. Khouider, An applied mathematics perspective on stochastic modelling for climate,, Phil. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 366 (2008), 2429.   Google Scholar

[4]

A. J. Majda, R. Abramov and M. Grote, "Information Theory and Stochastics for Multiscale Nonlinear Systems,", CRM Monograph Series, 25 (2005).   Google Scholar

[5]

I. Horenko, Finite element approach to clustering of multidimensional time series,, SIAM J. Sci. Comput., 32 (2010), 62.  doi: 10.1137/080715962.  Google Scholar

[6]

I. Horenko, On the identification of nonstationary factor models and their application to atmospheric data analysis,, J. Atmos. Sci., 67 (2010), 1559.  doi: 10.1175/2010JAS3271.1.  Google Scholar

[7]

D. Crommelin and E. Vanden-Eijnden, Subgrid-scale parameterization with conditional Markov chains,, J. Atmos. Sci., 65 (2008), 2661.  doi: 10.1175/2008JAS2566.1.  Google Scholar

[8]

A. J. Majda, C. Franzke and D. Crommelin, Normal forms for reduced stochastic climate models,, PNAS, 106 (2009), 3649.  doi: 10.1073/pnas.0900173106.  Google Scholar

[9]

A. J. Majda, C. Franzke, A. Fischer and D. Crommelin, Distinct metastable atmospheric regimes despite nearly Gaussian statistics: A paradigm model,, PNAS, 103 (2006), 8309.  doi: 10.1073/pnas.0602641103.  Google Scholar

[10]

C. Franzke, A. J. Majda and E. Vanden-Eijnden, Low-order stochastic mode reduction for a realistic barotropic model climate,, J. Atmos. Sci., 62 (2005), 1722.  doi: 10.1175/JAS3438.1.  Google Scholar

[11]

T. Delsole, Stochastic models of quasigeostrophic turbulence,, Surv. Geophys., 25 (2004), 107.   Google Scholar

[12]

A. J. Majda, J. Harlim and B. Gershgorin, Mathematical strategies for filtering turbulent dynamical systems,, Discrete and Continuous Dynamical Systems, 27 (2010), 441.  doi: 10.3934/dcds.2010.27.441.  Google Scholar

[13]

A. J. Majda, B. Gershgorin and Y. Yuan, Low frequency response and fluctuation-dissipation Theorems: Theory and practice,, J. Atmos. Sci., 67 (2010), 1186.  doi: 10.1175/2009JAS3264.1.  Google Scholar

[14]

A. J. Majda, R. Abramov and B. Gershgorin, High skill in low-frequency climate response through fluctuation dissipation theorems despite structural instability,, PNAS, 107 (2010), 581.  doi: 10.1073/pnas.0912997107.  Google Scholar

[15]

S. Kravtsov, D. Kondrashov and M. Ghil, Multilevel regression modeling of nonlinear processes: Derivation and applications to climatic variability,, J. Climate, 18 (2005), 4404.  doi: 10.1175/JCLI3544.1.  Google Scholar

[16]

D. Kondrashov, S. Kravtsov, A. W. Robertson and M. Ghil, A hierarchy of data-based ENSO models,, J. Climate, 18 (2005), 4425.  doi: 10.1175/JCLI3567.1.  Google Scholar

[17]

D. Kondrashov, S. Kravtsov and M. Ghil, Empirical mode reduction in a model of extratropical low-frequency variability,, J. Atmos. Sci, 63 (2006), 1859.  doi: 10.1175/JAS3719.1.  Google Scholar

[18]

G. A. Pavliotis and A. M. Stuart, Parameter estimation for multiscale diffusions,, J. Stat. Phys., 127 (2007), 741.  doi: 10.1007/s10955-007-9300-6.  Google Scholar

[19]

R. Azencott, A. Beri and I. Timofeyev, Adaptive sub-sampling for parametric estimation of Gaussian diffusions,, J. Stat. Phys., 139 (2010), 1066.  doi: 10.1007/s10955-010-9975-y.  Google Scholar

[20]

R. Azencott, A. Beri and I. Timofeyev, Sub-sampling and parametric estimation for multi-scale dynamics,, submitted, (2010).   Google Scholar

[21]

Y. Yuan and A. J. Majda, Invariant measures and asymptotic gaussian bounds for normal forms of stochastic climate model,, Chinese Annals of Mathematics, 32 (2011), 343.  doi: 10.1007/s11401-011-0647-2.  Google Scholar

[22]

A. J. Majda and B. Gershgorin, Quantifying uncertainty in climate change science through empirical information theory,, PNAS, 107 (2010), 14958.  doi: 10.1073/pnas.1007009107.  Google Scholar

[23]

C. Franzke, I. Horenko, A. J. Majda and R. Klein, Systematic metastable atmospheric regime identification in a AGCM,, J. Atmos. Sci., 66 (2009), 1997.  doi: 10.1175/2009JAS2939.1.  Google Scholar

[24]

J. Berner and G. Branstator, Linear and nonlinear signatures in planetary wave dynamics of an AGCM: Probability density functions,, J. Atmos. Sci., 64 (2007), 117.  doi: 10.1175/JAS3822.1.  Google Scholar

[25]

R. Z. Has'minskiĭ, A limit theorem for solutions of differential equations with a random right-hand part,, Teor. Verojatnost. i Primenen, 11 (1966), 444.   Google Scholar

[26]

T. G. Kurtz, A limit theorem for perturbed operator semigroups with applications to random evolutions,, J. Functional Analysis, 12 (1973), 55.  doi: 10.1016/0022-1236(73)90089-X.  Google Scholar

[1]

Alex H. Ardila, Mykael Cardoso. Blow-up solutions and strong instability of ground states for the inhomogeneous nonlinear Schrödinger equation. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020259

[2]

Tayeb Hadj Kaddour, Michael Reissig. Blow-up results for effectively damped wave models with nonlinear memory. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020239

[3]

María J. Cáceres, Ricarda Schneider. Blow-up, steady states and long time behaviour of excitatory-inhibitory nonlinear neuron models. Kinetic & Related Models, 2017, 10 (3) : 587-612. doi: 10.3934/krm.2017024

[4]

Jorge A. Esquivel-Avila. Blow-up in damped abstract nonlinear equations. Electronic Research Archive, 2020, 28 (1) : 347-367. doi: 10.3934/era.2020020

[5]

Victor A. Galaktionov, Juan-Luis Vázquez. The problem Of blow-up in nonlinear parabolic equations. Discrete & Continuous Dynamical Systems - A, 2002, 8 (2) : 399-433. doi: 10.3934/dcds.2002.8.399

[6]

José R. Quintero, Juan C. Cordero. Instability of the standing waves for a Benney-Roskes/Zakharov-Rubenchik system and blow-up for the Zakharov equations. Discrete & Continuous Dynamical Systems - B, 2020, 25 (4) : 1213-1240. doi: 10.3934/dcdsb.2019217

[7]

Mahdi Roozbeh, Saman Babaie–Kafaki, Zohre Aminifard. Two penalized mixed–integer nonlinear programming approaches to tackle multicollinearity and outliers effects in linear regression models. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020128

[8]

Jong-Shenq Guo. Blow-up behavior for a quasilinear parabolic equation with nonlinear boundary condition. Discrete & Continuous Dynamical Systems - A, 2007, 18 (1) : 71-84. doi: 10.3934/dcds.2007.18.71

[9]

Evgeny Galakhov, Olga Salieva. Blow-up for nonlinear inequalities with gradient terms and singularities on unbounded sets. Conference Publications, 2015, 2015 (special) : 489-494. doi: 10.3934/proc.2015.0489

[10]

Satyanad Kichenassamy. Control of blow-up singularities for nonlinear wave equations. Evolution Equations & Control Theory, 2013, 2 (4) : 669-677. doi: 10.3934/eect.2013.2.669

[11]

Pavol Quittner, Philippe Souplet. Blow-up rate of solutions of parabolic poblems with nonlinear boundary conditions. Discrete & Continuous Dynamical Systems - S, 2012, 5 (3) : 671-681. doi: 10.3934/dcdss.2012.5.671

[12]

Monica Marras, Stella Vernier Piro. Bounds for blow-up time in nonlinear parabolic systems. Conference Publications, 2011, 2011 (Special) : 1025-1031. doi: 10.3934/proc.2011.2011.1025

[13]

Van Duong Dinh. Blow-up criteria for linearly damped nonlinear Schrödinger equations. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020082

[14]

Dapeng Du, Yifei Wu, Kaijun Zhang. On blow-up criterion for the nonlinear Schrödinger equation. Discrete & Continuous Dynamical Systems - A, 2016, 36 (7) : 3639-3650. doi: 10.3934/dcds.2016.36.3639

[15]

Laurent Di Menza, Olivier Goubet. Stabilizing blow up solutions to nonlinear schrÖdinger equations. Communications on Pure & Applied Analysis, 2017, 16 (3) : 1059-1082. doi: 10.3934/cpaa.2017051

[16]

Walter A. Strauss, Kimitoshi Tsutaya. Existence and blow up of small amplitude nonlinear waves with a negative potential. Discrete & Continuous Dynamical Systems - A, 1997, 3 (2) : 175-188. doi: 10.3934/dcds.1997.3.175

[17]

Antonio Vitolo, Maria E. Amendola, Giulio Galise. On the uniqueness of blow-up solutions of fully nonlinear elliptic equations. Conference Publications, 2013, 2013 (special) : 771-780. doi: 10.3934/proc.2013.2013.771

[18]

Françoise Demengel, O. Goubet. Existence of boundary blow up solutions for singular or degenerate fully nonlinear equations. Communications on Pure & Applied Analysis, 2013, 12 (2) : 621-645. doi: 10.3934/cpaa.2013.12.621

[19]

Huiling Li, Mingxin Wang. Properties of blow-up solutions to a parabolic system with nonlinear localized terms. Discrete & Continuous Dynamical Systems - A, 2005, 13 (3) : 683-700. doi: 10.3934/dcds.2005.13.683

[20]

Filippo Gazzola, Paschalis Karageorgis. Refined blow-up results for nonlinear fourth order differential equations. Communications on Pure & Applied Analysis, 2015, 14 (2) : 677-693. doi: 10.3934/cpaa.2015.14.677

2019 Impact Factor: 1.27

Metrics

  • PDF downloads (66)
  • HTML views (0)
  • Cited by (18)

Other articles
by authors

[Back to Top]