June  2012, 17(4): 1333-1363. doi: 10.3934/dcdsb.2012.17.1333

Fundamental limitations of Ad hoc linear and quadratic multi-level regression models for physical systems

1. 

Department of Mathematics and Center for Atmosphere and Ocean Science, Courant Institute for Mathematical Sciences, New York University, New York, NY 10012-1110, United States

2. 

Courant Institute for Mathematical Sciences, New York University, New York, NY 10012-1110, United States

Received  November 2010 Revised  January 2012 Published  February 2012

A central issue in contemporary applied mathematics is the development of simpler dynamical models for a reduced subset of variables in complex high dimensional dynamical systems with many spatio-temporal scales. Recently, ad hoc quadratic multi-level regression models have been proposed to provide suitable reduced nonlinear models directly from data. The main results developed here are rigorous theorems demonstrating the non-physical finite time blow-up and large time instability in statistical solutions of general scalar multi-level quadratic regression models with corresponding unphysical features of the invariant measure. Surprising intrinsic model errors due to discrete sampling errors are also shown to occur rigorously even for linear multi-level regression dynamic models. all of these theoretical results are corroborated by numerical experiments with simple models. Single level nonlinear regression strategies with physical cubic damping are shown to have significant skill on the same test problems.
Citation: Andrew J. Majda, Yuan Yuan. Fundamental limitations of Ad hoc linear and quadratic multi-level regression models for physical systems. Discrete & Continuous Dynamical Systems - B, 2012, 17 (4) : 1333-1363. doi: 10.3934/dcdsb.2012.17.1333
References:
[1]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, Models for stochastic climate prediction,, PNAS, 96 (1999), 14687. doi: 10.1073/pnas.96.26.14687. Google Scholar

[2]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, A mathematical framework for stochastic climate models,, Commun. Pure Appl. Math., 54 (2001), 891. doi: 10.1002/cpa.1014. Google Scholar

[3]

A. J. Majda, C. Franzke and B. Khouider, An applied mathematics perspective on stochastic modelling for climate,, Phil. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 366 (2008), 2429. Google Scholar

[4]

A. J. Majda, R. Abramov and M. Grote, "Information Theory and Stochastics for Multiscale Nonlinear Systems,", CRM Monograph Series, 25 (2005). Google Scholar

[5]

I. Horenko, Finite element approach to clustering of multidimensional time series,, SIAM J. Sci. Comput., 32 (2010), 62. doi: 10.1137/080715962. Google Scholar

[6]

I. Horenko, On the identification of nonstationary factor models and their application to atmospheric data analysis,, J. Atmos. Sci., 67 (2010), 1559. doi: 10.1175/2010JAS3271.1. Google Scholar

[7]

D. Crommelin and E. Vanden-Eijnden, Subgrid-scale parameterization with conditional Markov chains,, J. Atmos. Sci., 65 (2008), 2661. doi: 10.1175/2008JAS2566.1. Google Scholar

[8]

A. J. Majda, C. Franzke and D. Crommelin, Normal forms for reduced stochastic climate models,, PNAS, 106 (2009), 3649. doi: 10.1073/pnas.0900173106. Google Scholar

[9]

A. J. Majda, C. Franzke, A. Fischer and D. Crommelin, Distinct metastable atmospheric regimes despite nearly Gaussian statistics: A paradigm model,, PNAS, 103 (2006), 8309. doi: 10.1073/pnas.0602641103. Google Scholar

[10]

C. Franzke, A. J. Majda and E. Vanden-Eijnden, Low-order stochastic mode reduction for a realistic barotropic model climate,, J. Atmos. Sci., 62 (2005), 1722. doi: 10.1175/JAS3438.1. Google Scholar

[11]

T. Delsole, Stochastic models of quasigeostrophic turbulence,, Surv. Geophys., 25 (2004), 107. Google Scholar

[12]

A. J. Majda, J. Harlim and B. Gershgorin, Mathematical strategies for filtering turbulent dynamical systems,, Discrete and Continuous Dynamical Systems, 27 (2010), 441. doi: 10.3934/dcds.2010.27.441. Google Scholar

[13]

A. J. Majda, B. Gershgorin and Y. Yuan, Low frequency response and fluctuation-dissipation Theorems: Theory and practice,, J. Atmos. Sci., 67 (2010), 1186. doi: 10.1175/2009JAS3264.1. Google Scholar

[14]

A. J. Majda, R. Abramov and B. Gershgorin, High skill in low-frequency climate response through fluctuation dissipation theorems despite structural instability,, PNAS, 107 (2010), 581. doi: 10.1073/pnas.0912997107. Google Scholar

[15]

S. Kravtsov, D. Kondrashov and M. Ghil, Multilevel regression modeling of nonlinear processes: Derivation and applications to climatic variability,, J. Climate, 18 (2005), 4404. doi: 10.1175/JCLI3544.1. Google Scholar

[16]

D. Kondrashov, S. Kravtsov, A. W. Robertson and M. Ghil, A hierarchy of data-based ENSO models,, J. Climate, 18 (2005), 4425. doi: 10.1175/JCLI3567.1. Google Scholar

[17]

D. Kondrashov, S. Kravtsov and M. Ghil, Empirical mode reduction in a model of extratropical low-frequency variability,, J. Atmos. Sci, 63 (2006), 1859. doi: 10.1175/JAS3719.1. Google Scholar

[18]

G. A. Pavliotis and A. M. Stuart, Parameter estimation for multiscale diffusions,, J. Stat. Phys., 127 (2007), 741. doi: 10.1007/s10955-007-9300-6. Google Scholar

[19]

R. Azencott, A. Beri and I. Timofeyev, Adaptive sub-sampling for parametric estimation of Gaussian diffusions,, J. Stat. Phys., 139 (2010), 1066. doi: 10.1007/s10955-010-9975-y. Google Scholar

[20]

R. Azencott, A. Beri and I. Timofeyev, Sub-sampling and parametric estimation for multi-scale dynamics,, submitted, (2010). Google Scholar

[21]

Y. Yuan and A. J. Majda, Invariant measures and asymptotic gaussian bounds for normal forms of stochastic climate model,, Chinese Annals of Mathematics, 32 (2011), 343. doi: 10.1007/s11401-011-0647-2. Google Scholar

[22]

A. J. Majda and B. Gershgorin, Quantifying uncertainty in climate change science through empirical information theory,, PNAS, 107 (2010), 14958. doi: 10.1073/pnas.1007009107. Google Scholar

[23]

C. Franzke, I. Horenko, A. J. Majda and R. Klein, Systematic metastable atmospheric regime identification in a AGCM,, J. Atmos. Sci., 66 (2009), 1997. doi: 10.1175/2009JAS2939.1. Google Scholar

[24]

J. Berner and G. Branstator, Linear and nonlinear signatures in planetary wave dynamics of an AGCM: Probability density functions,, J. Atmos. Sci., 64 (2007), 117. doi: 10.1175/JAS3822.1. Google Scholar

[25]

R. Z. Has'minskiĭ, A limit theorem for solutions of differential equations with a random right-hand part,, Teor. Verojatnost. i Primenen, 11 (1966), 444. Google Scholar

[26]

T. G. Kurtz, A limit theorem for perturbed operator semigroups with applications to random evolutions,, J. Functional Analysis, 12 (1973), 55. doi: 10.1016/0022-1236(73)90089-X. Google Scholar

show all references

References:
[1]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, Models for stochastic climate prediction,, PNAS, 96 (1999), 14687. doi: 10.1073/pnas.96.26.14687. Google Scholar

[2]

A. J. Majda, I. Timofeyev and E. Vanden Eijnden, A mathematical framework for stochastic climate models,, Commun. Pure Appl. Math., 54 (2001), 891. doi: 10.1002/cpa.1014. Google Scholar

[3]

A. J. Majda, C. Franzke and B. Khouider, An applied mathematics perspective on stochastic modelling for climate,, Phil. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 366 (2008), 2429. Google Scholar

[4]

A. J. Majda, R. Abramov and M. Grote, "Information Theory and Stochastics for Multiscale Nonlinear Systems,", CRM Monograph Series, 25 (2005). Google Scholar

[5]

I. Horenko, Finite element approach to clustering of multidimensional time series,, SIAM J. Sci. Comput., 32 (2010), 62. doi: 10.1137/080715962. Google Scholar

[6]

I. Horenko, On the identification of nonstationary factor models and their application to atmospheric data analysis,, J. Atmos. Sci., 67 (2010), 1559. doi: 10.1175/2010JAS3271.1. Google Scholar

[7]

D. Crommelin and E. Vanden-Eijnden, Subgrid-scale parameterization with conditional Markov chains,, J. Atmos. Sci., 65 (2008), 2661. doi: 10.1175/2008JAS2566.1. Google Scholar

[8]

A. J. Majda, C. Franzke and D. Crommelin, Normal forms for reduced stochastic climate models,, PNAS, 106 (2009), 3649. doi: 10.1073/pnas.0900173106. Google Scholar

[9]

A. J. Majda, C. Franzke, A. Fischer and D. Crommelin, Distinct metastable atmospheric regimes despite nearly Gaussian statistics: A paradigm model,, PNAS, 103 (2006), 8309. doi: 10.1073/pnas.0602641103. Google Scholar

[10]

C. Franzke, A. J. Majda and E. Vanden-Eijnden, Low-order stochastic mode reduction for a realistic barotropic model climate,, J. Atmos. Sci., 62 (2005), 1722. doi: 10.1175/JAS3438.1. Google Scholar

[11]

T. Delsole, Stochastic models of quasigeostrophic turbulence,, Surv. Geophys., 25 (2004), 107. Google Scholar

[12]

A. J. Majda, J. Harlim and B. Gershgorin, Mathematical strategies for filtering turbulent dynamical systems,, Discrete and Continuous Dynamical Systems, 27 (2010), 441. doi: 10.3934/dcds.2010.27.441. Google Scholar

[13]

A. J. Majda, B. Gershgorin and Y. Yuan, Low frequency response and fluctuation-dissipation Theorems: Theory and practice,, J. Atmos. Sci., 67 (2010), 1186. doi: 10.1175/2009JAS3264.1. Google Scholar

[14]

A. J. Majda, R. Abramov and B. Gershgorin, High skill in low-frequency climate response through fluctuation dissipation theorems despite structural instability,, PNAS, 107 (2010), 581. doi: 10.1073/pnas.0912997107. Google Scholar

[15]

S. Kravtsov, D. Kondrashov and M. Ghil, Multilevel regression modeling of nonlinear processes: Derivation and applications to climatic variability,, J. Climate, 18 (2005), 4404. doi: 10.1175/JCLI3544.1. Google Scholar

[16]

D. Kondrashov, S. Kravtsov, A. W. Robertson and M. Ghil, A hierarchy of data-based ENSO models,, J. Climate, 18 (2005), 4425. doi: 10.1175/JCLI3567.1. Google Scholar

[17]

D. Kondrashov, S. Kravtsov and M. Ghil, Empirical mode reduction in a model of extratropical low-frequency variability,, J. Atmos. Sci, 63 (2006), 1859. doi: 10.1175/JAS3719.1. Google Scholar

[18]

G. A. Pavliotis and A. M. Stuart, Parameter estimation for multiscale diffusions,, J. Stat. Phys., 127 (2007), 741. doi: 10.1007/s10955-007-9300-6. Google Scholar

[19]

R. Azencott, A. Beri and I. Timofeyev, Adaptive sub-sampling for parametric estimation of Gaussian diffusions,, J. Stat. Phys., 139 (2010), 1066. doi: 10.1007/s10955-010-9975-y. Google Scholar

[20]

R. Azencott, A. Beri and I. Timofeyev, Sub-sampling and parametric estimation for multi-scale dynamics,, submitted, (2010). Google Scholar

[21]

Y. Yuan and A. J. Majda, Invariant measures and asymptotic gaussian bounds for normal forms of stochastic climate model,, Chinese Annals of Mathematics, 32 (2011), 343. doi: 10.1007/s11401-011-0647-2. Google Scholar

[22]

A. J. Majda and B. Gershgorin, Quantifying uncertainty in climate change science through empirical information theory,, PNAS, 107 (2010), 14958. doi: 10.1073/pnas.1007009107. Google Scholar

[23]

C. Franzke, I. Horenko, A. J. Majda and R. Klein, Systematic metastable atmospheric regime identification in a AGCM,, J. Atmos. Sci., 66 (2009), 1997. doi: 10.1175/2009JAS2939.1. Google Scholar

[24]

J. Berner and G. Branstator, Linear and nonlinear signatures in planetary wave dynamics of an AGCM: Probability density functions,, J. Atmos. Sci., 64 (2007), 117. doi: 10.1175/JAS3822.1. Google Scholar

[25]

R. Z. Has'minskiĭ, A limit theorem for solutions of differential equations with a random right-hand part,, Teor. Verojatnost. i Primenen, 11 (1966), 444. Google Scholar

[26]

T. G. Kurtz, A limit theorem for perturbed operator semigroups with applications to random evolutions,, J. Functional Analysis, 12 (1973), 55. doi: 10.1016/0022-1236(73)90089-X. Google Scholar

[1]

María J. Cáceres, Ricarda Schneider. Blow-up, steady states and long time behaviour of excitatory-inhibitory nonlinear neuron models. Kinetic & Related Models, 2017, 10 (3) : 587-612. doi: 10.3934/krm.2017024

[2]

Victor A. Galaktionov, Juan-Luis Vázquez. The problem Of blow-up in nonlinear parabolic equations. Discrete & Continuous Dynamical Systems - A, 2002, 8 (2) : 399-433. doi: 10.3934/dcds.2002.8.399

[3]

José R. Quintero, Juan C. Cordero. Instability of the standing waves for a Benney-Roskes/Zakharov-Rubenchik system and blow-up for the zakharov equations. Discrete & Continuous Dynamical Systems - B, 2017, 22 (11) : 0-0. doi: 10.3934/dcdsb.2019217

[4]

Evgeny Galakhov, Olga Salieva. Blow-up for nonlinear inequalities with gradient terms and singularities on unbounded sets. Conference Publications, 2015, 2015 (special) : 489-494. doi: 10.3934/proc.2015.0489

[5]

Jong-Shenq Guo. Blow-up behavior for a quasilinear parabolic equation with nonlinear boundary condition. Discrete & Continuous Dynamical Systems - A, 2007, 18 (1) : 71-84. doi: 10.3934/dcds.2007.18.71

[6]

Satyanad Kichenassamy. Control of blow-up singularities for nonlinear wave equations. Evolution Equations & Control Theory, 2013, 2 (4) : 669-677. doi: 10.3934/eect.2013.2.669

[7]

Pavol Quittner, Philippe Souplet. Blow-up rate of solutions of parabolic poblems with nonlinear boundary conditions. Discrete & Continuous Dynamical Systems - S, 2012, 5 (3) : 671-681. doi: 10.3934/dcdss.2012.5.671

[8]

Monica Marras, Stella Vernier Piro. Bounds for blow-up time in nonlinear parabolic systems. Conference Publications, 2011, 2011 (Special) : 1025-1031. doi: 10.3934/proc.2011.2011.1025

[9]

Dapeng Du, Yifei Wu, Kaijun Zhang. On blow-up criterion for the nonlinear Schrödinger equation. Discrete & Continuous Dynamical Systems - A, 2016, 36 (7) : 3639-3650. doi: 10.3934/dcds.2016.36.3639

[10]

Laurent Di Menza, Olivier Goubet. Stabilizing blow up solutions to nonlinear schrÖdinger equations. Communications on Pure & Applied Analysis, 2017, 16 (3) : 1059-1082. doi: 10.3934/cpaa.2017051

[11]

Walter A. Strauss, Kimitoshi Tsutaya. Existence and blow up of small amplitude nonlinear waves with a negative potential. Discrete & Continuous Dynamical Systems - A, 1997, 3 (2) : 175-188. doi: 10.3934/dcds.1997.3.175

[12]

Antonio Vitolo, Maria E. Amendola, Giulio Galise. On the uniqueness of blow-up solutions of fully nonlinear elliptic equations. Conference Publications, 2013, 2013 (special) : 771-780. doi: 10.3934/proc.2013.2013.771

[13]

Françoise Demengel, O. Goubet. Existence of boundary blow up solutions for singular or degenerate fully nonlinear equations. Communications on Pure & Applied Analysis, 2013, 12 (2) : 621-645. doi: 10.3934/cpaa.2013.12.621

[14]

Filippo Gazzola, Paschalis Karageorgis. Refined blow-up results for nonlinear fourth order differential equations. Communications on Pure & Applied Analysis, 2015, 14 (2) : 677-693. doi: 10.3934/cpaa.2015.14.677

[15]

Huiling Li, Mingxin Wang. Properties of blow-up solutions to a parabolic system with nonlinear localized terms. Discrete & Continuous Dynamical Systems - A, 2005, 13 (3) : 683-700. doi: 10.3934/dcds.2005.13.683

[16]

Lili Du, Zheng-An Yao. Localization of blow-up points for a nonlinear nonlocal porous medium equation. Communications on Pure & Applied Analysis, 2007, 6 (1) : 183-190. doi: 10.3934/cpaa.2007.6.183

[17]

Türker Özsarı. Blow-up of solutions of nonlinear Schrödinger equations with oscillating nonlinearities. Communications on Pure & Applied Analysis, 2019, 18 (1) : 539-558. doi: 10.3934/cpaa.2019027

[18]

Maria Antonietta Farina, Monica Marras, Giuseppe Viglialoro. On explicit lower bounds and blow-up times in a model of chemotaxis. Conference Publications, 2015, 2015 (special) : 409-417. doi: 10.3934/proc.2015.0409

[19]

Yihong Du, Zongming Guo. The degenerate logistic model and a singularly mixed boundary blow-up problem. Discrete & Continuous Dynamical Systems - A, 2006, 14 (1) : 1-29. doi: 10.3934/dcds.2006.14.1

[20]

Ansgar Jüngel, Oliver Leingang. Blow-up of solutions to semi-discrete parabolic-elliptic Keller-Segel models. Discrete & Continuous Dynamical Systems - B, 2019, 24 (9) : 4755-4782. doi: 10.3934/dcdsb.2019029

2018 Impact Factor: 1.008

Metrics

  • PDF downloads (11)
  • HTML views (0)
  • Cited by (14)

Other articles
by authors

[Back to Top]