September  2020, 2(3): 207-255. doi: 10.3934/fods.2020011

Data-driven evolutions of critical points

1. 

Department of Mathematics, University of Vienna, Oskar-Morgenstern-Platz 1, 1090 Vienna, Austria

2. 

Department of Mathematics, Technical University of Munich, Boltzmannstrasse 3, 85748 Garching bei München, Germany

3. 

Institute of Mathematics and Scientific Computing, Karl-Franzens University of Graz, Heinrichstrasse 36/III, 8010 Graz, Austria

* Corresponding author

Published  August 2020

Fund Project: S.A. acknowledges the support of the DFG Transregio 109 "Discretization in Geometry and Dynamics". M.F. acknowledges the support of the DFG Project "Identification of Energies from Observation of Evolutions" and the DFG SPP 1962 "Non-smooth and Complementarity-based Distributed Parameter Systems: Simulation and Hierarchical Optimization". R.H. acknowledges the support of the DFG-FWF IGDK1754 "Optimization and Numerical Analysis for Partial Differential Equations with Nonsmooth Structures" and the hospitality of TUM during the preparation of this work

In this paper we are concerned with the learnability of energies from data obtained by observing time evolutions of their critical points starting at random initial equilibria. As a byproduct of our theoretical framework we introduce the novel concept of mean-field limit of critical point evolutions and of their energy balance as a new form of transport. We formulate the energy learning as a variational problem, minimizing the discrepancy of energy competitors from fulfilling the equilibrium condition along any trajectory of critical points originated at random initial equilibria. By $ \Gamma $-convergence arguments we prove the convergence of minimal solutions obtained from finite number of observations to the exact energy in a suitable sense. The abstract framework is actually fully constructive and numerically implementable. Hence, the approximation of the energy from a finite number of observations of past evolutions allows one to simulate further evolutions, which are fully data-driven. As we aim at a precise quantitative analysis, and to provide concrete examples of tractable solutions, we present analytic and numerical results on the reconstruction of an elastic energy for a one-dimensional model of thin nonlinear-elastic rod.

Citation: Stefano Almi, Massimo Fornasier, Richard Huber. Data-driven evolutions of critical points. Foundations of Data Science, 2020, 2 (3) : 207-255. doi: 10.3934/fods.2020011
References:
[1]

V. Agostiniani and R. Rossi, Singular vanishing-viscosity limits of gradient flows: The finite-dimensional case, J. Differential Equations, 263 (2017), 7815-7855.  doi: 10.1016/j.jde.2017.08.027.  Google Scholar

[2]

V. AgostinianiR. Rossi and G. Savaré, On the transversality conditions and their genericity, Rend. Circ. Mat. Palermo, 64 (2015), 101-116.  doi: 10.1007/s12215-014-0184-4.  Google Scholar

[3]

V. Agostiniani, R. Rossi, and G. Savaré, Singular vanishing-viscosity limits of gradient flows in Hilbert spaces, personal communication: in preparation, (2018). Google Scholar

[4]

V. AlbaniU. M. AscherX. Yang and J. P. Zubelli, Data driven recovery of local volatility surfaces, Inverse Probl. Imaging, 11 (2017), 799-823.  doi: 10.3934/ipi.2017038.  Google Scholar

[5]

L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford Mathematical Monographs. The Clarendon Press, Oxford University Press, New York, 2000.  Google Scholar

[6]

L. Ambrosio, N. Gigli and G. Savaré, Gradient flows in metric spaces and in the space of probability measures, Lectures in Mathematics ETH Zürich, Birkhäuser Verlag, Basel, second ed., 2008.  Google Scholar

[7]

M. BonginiM. FornasierM. Hansen and M. Maggioni, Inferring interaction rules from observations of evolutive systems Ⅰ: The variational approach, Math. Models Methods Appl. Sci., 27 (2017), 909-951.  doi: 10.1142/S0218202517500208.  Google Scholar

[8]

A. Braides, Γ-Convergence for Beginners, Oxford Lecture Series in Mathematics and its Applications, 22. Oxford University Press, Oxford, 2002. doi: 10.1093/acprof:oso/9780198507840.001.0001.  Google Scholar

[9]

T. Q. Chen, Y. Rubanova, J. Bettencourt and D. K. Duvenaud, Neural ordinary differential equations, Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett, eds., Curran Associates, Inc., 31 (2018), 6571–6583. Google Scholar

[10]

S. ContiS. Müller and M. Ortiz, Data-driven problems in elasticity, Arch. Ration. Mech. Anal., 229 (2018), 79-123.  doi: 10.1007/s00205-017-1214-0.  Google Scholar

[11]

S. Crépey, Calibration of the local volatility in a generalized Black-Scholes model using Tikhonov regularization, SIAM J. Math. Anal., 34 (2003), 1183-1206.  doi: 10.1137/S0036141001400202.  Google Scholar

[12]

T. S. Cubitt, J. Eisert and M. M. Wolf, Extracting dynamical equations from experimental data is NP hard, Phys. Rev. Lett., 108 (2012), 120503. doi: 10.1103/PhysRevLett.108.120503.  Google Scholar

[13]

G. Dal Maso, An Introduction to $\Gamma$-Convergence, vol. 8 of Progress in Nonlinear Differential Equations and their Applications, Birkhäuser Boston, Inc., Boston, MA, 1993. doi: 10.1007/978-1-4612-0327-8.  Google Scholar

[14]

W. E, A proposal on machine learning via dynamical systems, Commun. Math. Stat., 5 (2017), 1-11.  doi: 10.1007/s40304-017-0103-z.  Google Scholar

[15]

W. E, J. Han and Q. Li, A mean-field optimal control formulation of deep learning, Res. Math. Sci., 6 (2019), No. 10, 41 pp. doi: 10.1007/s40687-018-0172-y.  Google Scholar

[16]

H. Egger and H. W. Engl, Tikhonov regularization applied to the inverse problem of option pricing: Convergence analysis and rates, Inverse Problems, 21 (2005), 1027-1045.  doi: 10.1088/0266-5611/21/3/014.  Google Scholar

[17]

R. Gerstberger and P. Rentrop, Feedforward neural nets as discretization schemes for ODES and DAES, 7th ICCAM 96 Congress., J. Comput. Appl. Math., 82 (1997), 117-128.  doi: 10.1016/S0377-0427(97)00085-X.  Google Scholar

[18]

M. C. Grant and S. P. Boyd, Graph implementations for nonsmooth convex programs, Recent Advances in Learning and Control, V. Blondel, S. Boyd, and H. Kimura, eds., Lecture Notes in Control and Information Sciences, Springer-Verlag Limited, 371 (2008), 95–110. doi: 10.1007/978-1-84800-155-8_7.  Google Scholar

[19]

M. Grant and S. Boyd, CVX: Matlab Software for Disciplined Convex Programming, version 2.1., http://cvxr.com/cvx, Mar. 2014. Google Scholar

[20]

J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, vol. 42 of Applied Mathematical Sciences, Springer-Verlag, New York, 1983. doi: 10.1007/978-1-4612-1140-2.  Google Scholar

[21]

T. Kirchdoerfer and M. Ortiz, Data-driven computational mechanics, Comput. Methods Appl. Mech. Engrg., 304 (2016), 81-101.  doi: 10.1016/j.cma.2016.02.001.  Google Scholar

[22]

F. Lu, M. Maggioni and S. Tang, Learning interaction kernels in heterogeneous systems of agents from multiple trajectories, arXiv: 1910.04832. Google Scholar

[23]

F. LuM. ZhongS. Tang and M. Maggioni, Nonparametric inference of interaction laws in systems of agents from trajectory data, Proc. Natl. Acad. Sci. USA, 116 (2019), 14424-14433.  doi: 10.1073/pnas.1822012116.  Google Scholar

[24]

E. Novak and H. Woźniakowski, Approximation of infinitely differentiable multivariate functions is intractable, J. Complexity, 25 (2009), 398-404.  doi: 10.1016/j.jco.2008.11.002.  Google Scholar

[25]

H. SchaefferG. Tran and R. Ward, Extracting sparse high-dimensional dynamics from limited data, SIAM J. Appl. Math., 78 (2018), 3279-3295.  doi: 10.1137/18M116798X.  Google Scholar

[26]

H. Schaeffer, G. Tran and R. Ward, Learning dynamical systems and bifurcation via group sparsity, arXiv: 1709.01558. Google Scholar

[27]

G. Scilla and F. Solombrino, Delayed loss of stability in singularly perturbed finite-dimensional gradient flows, Asymptot. Anal., 110 (2018), 1-19.  doi: 10.3233/ASY-181475.  Google Scholar

[28]

G. Scilla and F. Solombrino, Multiscale analysis of singularly perturbed finite dimensional gradient flows: The minimizing movement approach, Nonlinearity, 31 (2018), 5036-5074.  doi: 10.1088/1361-6544/aad6ac.  Google Scholar

[29]

G. Tran and R. Ward, Exact recovery of chaotic systems from highly corrupted data, Multiscale Model. Simul., 15 (2017), 1108-1129.  doi: 10.1137/16M1086637.  Google Scholar

[30]

C. Zanini, Singular perturbations of finite dimensional gradient flows, Discrete Contin. Dyn. Syst., 18 (2007), 657-675.  doi: 10.3934/dcds.2007.18.657.  Google Scholar

[31]

M. Zhong, J. Miller and M. Maggioni, Data-driven discovery of emergent behaviors in collective dynamics, Phys. D, 411 (2020), 132542, 25 pp. doi: 10.1016/j.physd.2020.132542.  Google Scholar

show all references

References:
[1]

V. Agostiniani and R. Rossi, Singular vanishing-viscosity limits of gradient flows: The finite-dimensional case, J. Differential Equations, 263 (2017), 7815-7855.  doi: 10.1016/j.jde.2017.08.027.  Google Scholar

[2]

V. AgostinianiR. Rossi and G. Savaré, On the transversality conditions and their genericity, Rend. Circ. Mat. Palermo, 64 (2015), 101-116.  doi: 10.1007/s12215-014-0184-4.  Google Scholar

[3]

V. Agostiniani, R. Rossi, and G. Savaré, Singular vanishing-viscosity limits of gradient flows in Hilbert spaces, personal communication: in preparation, (2018). Google Scholar

[4]

V. AlbaniU. M. AscherX. Yang and J. P. Zubelli, Data driven recovery of local volatility surfaces, Inverse Probl. Imaging, 11 (2017), 799-823.  doi: 10.3934/ipi.2017038.  Google Scholar

[5]

L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford Mathematical Monographs. The Clarendon Press, Oxford University Press, New York, 2000.  Google Scholar

[6]

L. Ambrosio, N. Gigli and G. Savaré, Gradient flows in metric spaces and in the space of probability measures, Lectures in Mathematics ETH Zürich, Birkhäuser Verlag, Basel, second ed., 2008.  Google Scholar

[7]

M. BonginiM. FornasierM. Hansen and M. Maggioni, Inferring interaction rules from observations of evolutive systems Ⅰ: The variational approach, Math. Models Methods Appl. Sci., 27 (2017), 909-951.  doi: 10.1142/S0218202517500208.  Google Scholar

[8]

A. Braides, Γ-Convergence for Beginners, Oxford Lecture Series in Mathematics and its Applications, 22. Oxford University Press, Oxford, 2002. doi: 10.1093/acprof:oso/9780198507840.001.0001.  Google Scholar

[9]

T. Q. Chen, Y. Rubanova, J. Bettencourt and D. K. Duvenaud, Neural ordinary differential equations, Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett, eds., Curran Associates, Inc., 31 (2018), 6571–6583. Google Scholar

[10]

S. ContiS. Müller and M. Ortiz, Data-driven problems in elasticity, Arch. Ration. Mech. Anal., 229 (2018), 79-123.  doi: 10.1007/s00205-017-1214-0.  Google Scholar

[11]

S. Crépey, Calibration of the local volatility in a generalized Black-Scholes model using Tikhonov regularization, SIAM J. Math. Anal., 34 (2003), 1183-1206.  doi: 10.1137/S0036141001400202.  Google Scholar

[12]

T. S. Cubitt, J. Eisert and M. M. Wolf, Extracting dynamical equations from experimental data is NP hard, Phys. Rev. Lett., 108 (2012), 120503. doi: 10.1103/PhysRevLett.108.120503.  Google Scholar

[13]

G. Dal Maso, An Introduction to $\Gamma$-Convergence, vol. 8 of Progress in Nonlinear Differential Equations and their Applications, Birkhäuser Boston, Inc., Boston, MA, 1993. doi: 10.1007/978-1-4612-0327-8.  Google Scholar

[14]

W. E, A proposal on machine learning via dynamical systems, Commun. Math. Stat., 5 (2017), 1-11.  doi: 10.1007/s40304-017-0103-z.  Google Scholar

[15]

W. E, J. Han and Q. Li, A mean-field optimal control formulation of deep learning, Res. Math. Sci., 6 (2019), No. 10, 41 pp. doi: 10.1007/s40687-018-0172-y.  Google Scholar

[16]

H. Egger and H. W. Engl, Tikhonov regularization applied to the inverse problem of option pricing: Convergence analysis and rates, Inverse Problems, 21 (2005), 1027-1045.  doi: 10.1088/0266-5611/21/3/014.  Google Scholar

[17]

R. Gerstberger and P. Rentrop, Feedforward neural nets as discretization schemes for ODES and DAES, 7th ICCAM 96 Congress., J. Comput. Appl. Math., 82 (1997), 117-128.  doi: 10.1016/S0377-0427(97)00085-X.  Google Scholar

[18]

M. C. Grant and S. P. Boyd, Graph implementations for nonsmooth convex programs, Recent Advances in Learning and Control, V. Blondel, S. Boyd, and H. Kimura, eds., Lecture Notes in Control and Information Sciences, Springer-Verlag Limited, 371 (2008), 95–110. doi: 10.1007/978-1-84800-155-8_7.  Google Scholar

[19]

M. Grant and S. Boyd, CVX: Matlab Software for Disciplined Convex Programming, version 2.1., http://cvxr.com/cvx, Mar. 2014. Google Scholar

[20]

J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, vol. 42 of Applied Mathematical Sciences, Springer-Verlag, New York, 1983. doi: 10.1007/978-1-4612-1140-2.  Google Scholar

[21]

T. Kirchdoerfer and M. Ortiz, Data-driven computational mechanics, Comput. Methods Appl. Mech. Engrg., 304 (2016), 81-101.  doi: 10.1016/j.cma.2016.02.001.  Google Scholar

[22]

F. Lu, M. Maggioni and S. Tang, Learning interaction kernels in heterogeneous systems of agents from multiple trajectories, arXiv: 1910.04832. Google Scholar

[23]

F. LuM. ZhongS. Tang and M. Maggioni, Nonparametric inference of interaction laws in systems of agents from trajectory data, Proc. Natl. Acad. Sci. USA, 116 (2019), 14424-14433.  doi: 10.1073/pnas.1822012116.  Google Scholar

[24]

E. Novak and H. Woźniakowski, Approximation of infinitely differentiable multivariate functions is intractable, J. Complexity, 25 (2009), 398-404.  doi: 10.1016/j.jco.2008.11.002.  Google Scholar

[25]

H. SchaefferG. Tran and R. Ward, Extracting sparse high-dimensional dynamics from limited data, SIAM J. Appl. Math., 78 (2018), 3279-3295.  doi: 10.1137/18M116798X.  Google Scholar

[26]

H. Schaeffer, G. Tran and R. Ward, Learning dynamical systems and bifurcation via group sparsity, arXiv: 1709.01558. Google Scholar

[27]

G. Scilla and F. Solombrino, Delayed loss of stability in singularly perturbed finite-dimensional gradient flows, Asymptot. Anal., 110 (2018), 1-19.  doi: 10.3233/ASY-181475.  Google Scholar

[28]

G. Scilla and F. Solombrino, Multiscale analysis of singularly perturbed finite dimensional gradient flows: The minimizing movement approach, Nonlinearity, 31 (2018), 5036-5074.  doi: 10.1088/1361-6544/aad6ac.  Google Scholar

[29]

G. Tran and R. Ward, Exact recovery of chaotic systems from highly corrupted data, Multiscale Model. Simul., 15 (2017), 1108-1129.  doi: 10.1137/16M1086637.  Google Scholar

[30]

C. Zanini, Singular perturbations of finite dimensional gradient flows, Discrete Contin. Dyn. Syst., 18 (2007), 657-675.  doi: 10.3934/dcds.2007.18.657.  Google Scholar

[31]

M. Zhong, J. Miller and M. Maggioni, Data-driven discovery of emergent behaviors in collective dynamics, Phys. D, 411 (2020), 132542, 25 pp. doi: 10.1016/j.physd.2020.132542.  Google Scholar

Figure 1.  Reconstructions $ \hat a $ (left) and $ \hat a' $ (right) in dotted red and true $ a $ and $ a' $ in blue. The underlying red circles depict the (adaptive) nodes in $ \Lambda $. Below you see the distribution $ \tilde \eta $ of available data
Figure 2.  Reconstruction of $ a' $ with increasing number $ {N_e} $ of measurements. The true $ a' $ is depicted by the blue curve while the red line depicts the reconstruction $ \hat a' $ following (5.23). The red circles at the bottom of the figures depict the position of nodes of the underlying mesh $ \Lambda_N $. The histogram below describes the density of the available information encoded by the probability measure $ \tilde{\eta} $ for the case $ N_e = 60 $
Figure 3.  Reconstruction of $ a' $ for varying $ N $ and $ D(N) = 4N $ and distribution $ \tilde \eta $ for $ N = 60 $. Graphic as described in Figure 2
Figure 4.  Reconstruction of $ a' $ for varying $ N $ and fixed $ D(N) = 300 $ and distribution $ \tilde \eta $. Graphics as described in Figure 2
Figure 5.  Impact of different constants $ M_2 $ for constraints on $ a'' $ with $ \|a''\| \leq [2,5;20,1000] $ on reconstructs are depicted, showing improved approximations for increasing $ M_2 $. Graphics as in Figure 2
Figure 6.  Impact of changing constraint $ M_1 $ bounding the values of $ a' $, showing a projection-like behaviour of the reconstruction for small $ M_1 $. Graphics as in Figure 2
Figure 7.  Left showing $ \hat x_\varepsilon(t) $ (dark dashed) and $ x_\varepsilon(t) $ (line bright) with data from Section 5.2.2 at times $ t = [0.2 $, $ 0.4 $, $ 0.6 $, $ 0.8 $, $ 1.0] $, right with data from Section 5.2.3
[1]

Illés Horváth, Kristóf Attila Horváth, Péter Kovács, Miklós Telek. Mean-field analysis of a scaling MAC radio protocol. Journal of Industrial & Management Optimization, 2021, 17 (1) : 279-297. doi: 10.3934/jimo.2019111

[2]

Theresa Lange, Wilhelm Stannat. Mean field limit of ensemble square root filters - discrete and continuous time. Foundations of Data Science, 2021  doi: 10.3934/fods.2021003

[3]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control & Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026

[4]

Xinfu Chen, Huiqiang Jiang, Guoqing Liu. Boundary spike of the singular limit of an energy minimizing problem. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3253-3290. doi: 10.3934/dcds.2020124

[5]

Geir Evensen, Javier Amezcua, Marc Bocquet, Alberto Carrassi, Alban Farchi, Alison Fowler, Pieter L. Houtekamer, Christopher K. Jones, Rafael J. de Moraes, Manuel Pulido, Christian Sampson, Femke C. Vossepoel. An international initiative of predicting the SARS-CoV-2 pandemic using ensemble data assimilation. Foundations of Data Science, 2020  doi: 10.3934/fods.2021001

[6]

Laura Aquilanti, Simone Cacace, Fabio Camilli, Raul De Maio. A Mean Field Games model for finite mixtures of Bernoulli and categorical distributions. Journal of Dynamics & Games, 2020  doi: 10.3934/jdg.2020033

[7]

Jianfeng Huang, Haihua Liang. Limit cycles of planar system defined by the sum of two quasi-homogeneous vector fields. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 861-873. doi: 10.3934/dcdsb.2020145

[8]

Marion Darbas, Jérémy Heleine, Stephanie Lohrengel. Numerical resolution by the quasi-reversibility method of a data completion problem for Maxwell's equations. Inverse Problems & Imaging, 2020, 14 (6) : 1107-1133. doi: 10.3934/ipi.2020056

[9]

Kai Yang. Scattering of the focusing energy-critical NLS with inverse square potential in the radial case. Communications on Pure & Applied Analysis, 2021, 20 (1) : 77-99. doi: 10.3934/cpaa.2020258

[10]

Daniele Bartolucci, Changfeng Gui, Yeyao Hu, Aleks Jevnikar, Wen Yang. Mean field equations on tori: Existence and uniqueness of evenly symmetric blow-up solutions. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3093-3116. doi: 10.3934/dcds.2020039

[11]

Alain Bensoussan, Xinwei Feng, Jianhui Huang. Linear-quadratic-Gaussian mean-field-game with partial observation and common noise. Mathematical Control & Related Fields, 2021, 11 (1) : 23-46. doi: 10.3934/mcrf.2020025

[12]

Dan Zhu, Rosemary A. Renaut, Hongwei Li, Tianyou Liu. Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory. Inverse Problems & Imaging, 2021, 15 (1) : 159-183. doi: 10.3934/ipi.2020076

[13]

Manuel del Pino, Monica Musso, Juncheng Wei, Yifu Zhou. Type Ⅱ finite time blow-up for the energy critical heat equation in $ \mathbb{R}^4 $. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3327-3355. doi: 10.3934/dcds.2020052

[14]

Gernot Holler, Karl Kunisch. Learning nonlocal regularization operators. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021003

[15]

Fabian Ziltener. Note on coisotropic Floer homology and leafwise fixed points. Electronic Research Archive, , () : -. doi: 10.3934/era.2021001

[16]

Guo-Niu Han, Huan Xiong. Skew doubled shifted plane partitions: Calculus and asymptotics. Electronic Research Archive, 2021, 29 (1) : 1841-1857. doi: 10.3934/era.2020094

[17]

Thomas Y. Hou, Dong Liang. Multiscale analysis for convection dominated transport equations. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 281-298. doi: 10.3934/dcds.2009.23.281

[18]

Bing Yu, Lei Zhang. Global optimization-based dimer method for finding saddle points. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 741-753. doi: 10.3934/dcdsb.2020139

[19]

Andrew D. Lewis. Erratum for "nonholonomic and constrained variational mechanics". Journal of Geometric Mechanics, 2020, 12 (4) : 671-675. doi: 10.3934/jgm.2020033

[20]

Jie Li, Xiangdong Ye, Tao Yu. Mean equicontinuity, complexity and applications. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 359-393. doi: 10.3934/dcds.2020167

 Impact Factor: 

Article outline

Figures and Tables

[Back to Top]