\`x^2+y_1+z_12^34\`
Advanced Search
Advanced Search
Advanced Search
This issue Previous Article Next Article
Article Contents
Article Contents
2025, Volume 7, Issue 1: 298-337. Doi: 10.3934/fods.2024043
This issue Previous Article Multifidelity linear regression for scientific machine learning from scarce data Next Article Reduced basis approximations of parameterized dynamical partial differential equations via neural networks

Scientific machine learning for closure models in multiscale problems: A review

  • 1.

    Scientific Computing Group, Centrum Wiskunde & Informatica, Science Park 123, Amsterdam, The Netherlands

  • 2.

    Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, 99352, WA, USA

  • 3.

    College of Information Sciences and Technology, The Pennsylvania State University, Westgate Building, University Park, 16802, PA, USA

  • *Corresponding author: Benjamin Sanderse

    *Corresponding author: Benjamin Sanderse 
Received:  March 2024
Revised:  August 2024
Early access:  October 2024
Published:  March 2025
Abstract / Introduction Full Text(HTML) Figure(3) Related Papers Cited by
    Abstract

  • Closure problems are omnipresent when simulating multiscale systems, where some quantities and processes cannot be fully prescribed despite their effects on the simulation's accuracy. Recently, scientific machine learning approaches have been proposed as a way to tackle the closure problem, combining traditional (physics-based) modeling with data-driven (machine-learned) techniques, typically through enriching differential equations with neural networks. This paper reviews the different reduced model forms, distinguished by the degree to which they include known physics, and the different objectives of a priori and a posteriori learning. The importance of adhering to physical laws (such as symmetries and conservation laws) in choosing the reduced model form and choosing the learning method is discussed. The effect of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. In addition, we make the connections between closure problems and several other research disciplines: inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. In conclusion, much progress has been made with scientific machine learning approaches for solving closure problems, but many challenges remain. In particular, the generalizability and interpretability of learned models is a major issue that needs to be addressed further.

    Mathematics Subject Classification: Primary: 65MXX, 76MXX; Secondary: 68T07.

    Citation:
    shu

    \begin{equation} \\ \end{equation}
    • Full Text(HTML)
  • 加载中

  • Figure 1.  A priori vs. a posteriori learning. Training data $ \mathit{\boldsymbol{{u}}} $ and $ \bar{\mathit{\boldsymbol{{u}}}} $ are obtained from solving the high-fidelity model $ \mathit{\boldsymbol{{F}}} $. In blue: simulation of the high-fidelity model, from which the ground truth $ \bar{\mathit{\boldsymbol{{u}}}} $ is derived. In yellow: training. In a posteriori learning, the loss function (24) includes evaluating the gradients of the reduced model, while in a priori learning, only the residual must be evaluated (equation (22))

    Download: Full-size image PowerPoint slide

    Figure 2.  An example of imposing mass conservation for a 2D incompressible flow as a soft constraint (left) and hard constraint (right). For the hard constraint, the streamfunction $ \psi $ is learned instead of the velocity field; the velocity field follows as $ \boldsymbol{{u}} = \nabla \times \psi $, which by construction satisfies $ \nabla \cdot \boldsymbol{{u}} = 0 $

    Download: Full-size image PowerPoint slide

    Figure 3.  Optimize-then-discretize vs. discretize-then-optimize approaches. In orange: the optimization of a neural network to learn a time-continuous closure (left) or a time-discrete correction (right). In blue: the time integration/discretization step

    Download: Full-size image PowerPoint slide
    • Related Papers
  • Cited by
  • References
  • [1] A. Abdulle, E. Weinan, B. Engquist and E. Vanden-Eijnden, The heterogeneous multiscale method, Acta Numerica, 21 (2012), 1-87.  doi: 10.1017/S0962492912000025.
    [2] S. E. Ahmed, S. Pawar, O. San, A. Rasheed, T. Iliescu and B. R. Noack, On closures for reduced order models—a spectrum of first-principle to machine-learned avenues, Physics of Fluids, 33.
    [3] S. E. Ahmed, O. San, A. Rasheed, T. Iliescu and A. Veneziani, Physics guided machine learning for variational multiscale reduced order modeling, SIAM Journal on Scientific Computing, 45 (2023), B283-B313. doi: 10.1137/22M1496360.
    [4] S. E. Ahmed and P. Stinis, A multifidelity deep operator network approach to closure for multiscale systems, Computer Methods in Applied Mechanics and Engineering, 414 (2023), 116161.  doi: 10.1016/j.cma.2023.116161.
    [5] H. Arbabi, J. E. Bunder, G. Samaey, A. J. Roberts and I. G. Kevrekidis, Linking machine learning with multiscale numerics: Data-driven discovery of homogenized equations, Jom, 72 (2020), 4444-4457.  doi: 10.1007/s11837-020-04399-8.
    [6] G. Arranz, Y. Ling, S. Costa, K. Goc and A. Lozano-Duran, Building-block flow model for computational fluids, 2024.
    [7] M. Asch, M. Bocquet and M. Nodet, Data Assimilation: Methods, Algorithms, and Applications, Society for Industrial and Applied Mathematics, Philadelphia, PA, 2016.
    [8] H. J. Bae and P. Koumoutsakos, Scientific multi-agent reinforcement learning for wall-models of turbulent flows, Nature Communications, 13 (2022), 1443.  doi: 10.1038/s41467-022-28957-7.
    [9] Y. Bar-Sinai, S. Hoyer, J. Hickey and M. P. Brenner, Learning data-driven discretizations for partial differential equations, Proceedings of the National Academy of Sciences, 116 (2019), 15344-15349.  doi: 10.1073/pnas.1814058116.
    [10] F. Bartolucci, E. de Bezenac, B. Raonic, R. Molinaro, S. Mishra and R. Alaifari, Representation equivalent neural operators: A framework for alias-free operator learning, in Thirty-Seventh Conference on Neural Information Processing Systems, 2023.
    [11] A. Beck, D. Flad and C.-D. Munz, Deep neural networks for data-driven LES closure models, Journal of Computational Physics, 398 (2019), 108910.  doi: 10.1016/j.jcp.2019.108910.
    [12] A. Beck and M. Kurz, A perspective on machine learning methods in turbulence modeling, GAMM-Mitteilungen, 44 (2021), 27 pp.
    [13] A. Beck and M. Kurz, Toward discretization-consistent closure schemes for large eddy simulation using reinforcement learning, Physics of Fluids, 35.
    [14] J. Berner, U. Achatz, L. Batté, L. Bengtsson, A. de la Cámara, H. M. Christensen, M. Colangeli, D. R. B. Coleman, D. Crommelin, S. I. Dolaptchiev, C. L. E. Franzke, P. Friederichs, P. Imkeller, H. Järvinen, S. Juricke, V. Kitsios, F. Lott, V. Lucarini, S. Mahajan, T. N. Palmer, C. Penland, M. Sakradzija, J.-S. von Storch, A. Weisheimer, M. Weniger, P. D. Williams and J.-I. Yano, Stochastic Parameterization: Toward a New View of Weather and Climate Models, Bulletin of the American Meteorological Society, 98 (2017), 565-588. 
    [15] D. Bernstein, Optimal prediction of Burgers's equation, Multiscale Modeling Simulation, 6 (2007), 27-52. 
    [16] L. C. Berselli, T. Iliescu and W. J. Layton, Mathematics of Large Eddy Simulation of Turbulent Flows, Springer, 2006.
    [17] T. Beucler, S. Rasp, M. Pritchard and P. Gentine, Achieving conservation of energy in neural network emulators for climate modeling, 2019.
    [18] M. A. Bhouri and P. Gentine, History-based, bayesian, closure for stochastic parameterization: Application to lorenz'96, preprint, arXiv: 2210.14488.
    [19] M. A. Bhouri and P. Gentine, Memory-based parameterization with differentiable solver: Application to Lorenz '96, Chaos: An Interdisciplinary Journal of Nonlinear Science, 33 (2023), 073116.  doi: 10.1063/5.0131929.
    [20] M. Bocquet, J. Brajard, A. Carrassi and L. Bertino, Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization, Foundations of Data Science, 2 (2020), 55-80. 
    [21] C. Bodnar, W. P. Bruinsma, A. Lucic, M. Stanley, J. Brandstetter, P. Garvan, M. Riechert, J. Weyn, H. Dong, A. Vaughan, J. K. Gupta, K. Tambiratnam, A. Archibald, E. Heider, M. Welling, R. E. Turner and P. Perdikaris, Aurora: A Foundation Model of the Atmosphere, 2024.
    [22] T. Bolton and L. Zanna, Applications of deep learning to ocean data inference and subgrid parameterization, Journal of Advances in Modeling Earth Systems, 11 (2019), 376-399.  doi: 10.1029/2018MS001472.
    [23] A. Boral, Z. Y. Wan, L. Zepeda-Núñez, J. Lottes, Q. Wang, Y.-f. Chen, J. R. Anderson and F. Sha, Neural ideal large eddy simulation: Modeling turbulence with neural stochastic differential equations, 2023.
    [24] R. Bose and A. M. Roy, Invariance embedded physics-infused deep neural network-based sub-grid scale models for turbulent flows, Engineering Applications of Artificial Intelligence, 128 (2024), 107483.  doi: 10.1016/j.engappai.2023.107483.
    [25] J. Brandstetter, M. Welling and D. E. Worrall, Lie point symmetry data augmentation for neural pde solvers, in International Conference on Machine Learning, PMLR, (2022), 2241-2256.
    [26] J. Brandstetter, D. Worrall and M. Welling, Message Passing Neural PDE Solvers, 2023.
    [27] C. Brennan and D. Venturi, Data-driven closures for stochastic dynamical systems, Journal of Computational Physics, 372 (2018), 281-298.  doi: 10.1016/j.jcp.2018.06.038.
    [28] S. L. Brunton and J. N. Kutz, Machine learning for partial differential equations, 2023.
    [29] S. L. Brunton, J. L. Proctor and J. N. Kutz, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proceedings of the National Academy of Sciences, 113 (2016), 3932-3937.  doi: 10.1073/pnas.1517384113.
    [30] S. Cai, Z. Mao, Z. Wang, M. Yin and G. E. Karniadakis, Physics-informed neural networks (PINNs) for fluid mechanics: A review, Acta Mechanica Sinica, 37 (2021), 1727-1738.  doi: 10.1007/s10409-021-01148-1.
    [31] J. L. Callaham, S. L. Brunton and J.-C. Loiseau, On the role of nonlinear correlations in reduced-order modelling, Journal of Fluid Mechanics, 938 (2022), A1. doi: 10.1017/jfm.2021.994.
    [32] J. L. Callaham, J.-C. Loiseau and S. L. Brunton, Multiscale model reduction for incompressible flows, Journal of Fluid Mechanics, 973 (2023), A3. doi: 10.1017/jfm.2023.510.
    [33] D. Calvetti and E. Somersalo, Inverse problems: From regularization to Bayesian inference, WIREs Computational Statistics, 10 (2018), 19 pp. doi: 10.1002/wics.1427.
    [34] Q. Cao, S. Goswami and G. E. Karniadakis, Lno: Laplace neural operator for solving differential equations, preprint, arXiv: 2303.10528.
    [35] A.-T. G. Charalampopoulos and T. P. Sapsis, Machine-learning energy-preserving nonlocal closures for turbulent fluid flows and inertial tracers, Physical Review Fluids, 7 (2022), 024305.  doi: 10.1103/PhysRevFluids.7.024305.
    [36] M. D. Chekroun, H. Liu, K. Srinivasan and J. C. McWilliams, The high-frequency and rare events barriers to neural closures of atmospheric dynamics, Journal of Physics: Complexity, 5 (2024), 025004.  doi: 10.1088/2632-072X/ad3e59.
    [37] R. T. Q. Chen, Y. Rubanova, J. Bettencourt and D. Duvenaud, Neural ordinary differential equations, arXiv: 1806.07366.
    [38] X. Chen, J. Lu and G. Tryggvason, Finding closure models to match the time evolution of coarse grained 2d turbulence flows using machine learning, Fluids, 7 (2022), 154.  doi: 10.3390/fluids7050154.
    [39] A. J. Chorin, O. H. Hald and R. Kupferman, Optimal prediction and the Mori-Zwanzig representation of irreversible processes, Proceedings of the National Academy of Sciences, 97 (2000), 2968-2973.  doi: 10.1073/pnas.97.7.2968.
    [40] A. J. Chorin, O. H. Hald and R. Kupferman, Optimal prediction with memory, Physica D: Nonlinear Phenomena, 166 (2002), 239-257.  doi: 10.1016/S0167-2789(02)00446-3.
    [41] A. Chorin and P. Stinis, Problem reduction, renormalization, and memory, Communications in Applied Mathematics and Computational Science, 1 (2006), 1-27.  doi: 10.2140/camcos.2006.1.1.
    [42] M. Christie, Upscaling for reservoir simulation, J Pet Technol, 48 (1996), 1004-1010.  doi: 10.2118/37324-JPT.
    [43] H. Christensen and  L. Zanna,  Parametrization in Weather and Climate Models, Oxford University Press, 2022. 
    [44] P. Constantin, C. Foias, B. Nicolaenko and R. Temam, Integral Manifolds and Inertial Manifolds for Dissipative Partial Differential Equations, vol. 70, Springer Science & Business Media, 1989.
    [45] P. Conti, M. Guo, A. Manzoni, A. Frangi, S. L. Brunton and J. Nathan Kutz, Multi-fidelity reduced-order surrogate modelling, Proceedings of the Royal Society A, 480 (2024), 22 pp. doi: 10.1098/rspa.2023.0655.
    [46] P. Conti, M. Guo, A. Manzoni and J. S. Hesthaven, Multi-fidelity surrogate modeling using long short-term memory networks, Computer Methods in Applied Mechanics and Engineering, 404 (2023), 115811.  doi: 10.1016/j.cma.2022.115811.
    [47] E. R. Crabtree, J. M. Bello-Rivas, A. L. Ferguson and I. G. Kevrekidis, Gans and closures: Micro-macro consistency in multiscale modeling, Multiscale Modeling Simulation, 21 (2023), 1122-1146. 
    [48] A. J. Crilly, B. Duhig and N. Bouziani, Learning closure relations using differentiable programming: An example in radiation transport, Journal of Quantitative Spectroscopy and Radiative Transfer, 108941.
    [49] D. Crommelin and W. Edeling, Resampling with neural networks for stochastic parameterization in multiscale systems, Physica D: Nonlinear Phenomena, 422 (2021), 132894.  doi: 10.1016/j.physd.2021.132894.
    [50] E. A. B. de Moraes, M. D'Elia and M. Zayernouri, Machine learning of nonlocal micro-structural defect evolutions in crystalline materials, Computer Methods in Applied Mechanics and Engineering, 403 (2023), 115743.  doi: 10.1016/j.cma.2022.115743.
    [51] N. Demo, M. Tezzele and G. Rozza, A DeepONetmulti-fidelity approach for residual learning in reduced order modeling, preprint, arXiv: 2302.12682.
    [52] F. Dietrich, A. Makeev, G. Kevrekidis, N. Evangelou, T. Bertalan, S. Reich and I. G. Kevrekidis, Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning, Chaos: An Interdisciplinary Journal of Nonlinear Science, 33 (2023), 19 pp. doi: 10.1063/5.0113632.
    [53] G. Dresdner, D. Kochkov, P. Norgaard, L. Zepeda-Núñez, J. A. Smith, M. P. Brenner and S. Hoyer, Learning to correct spectral methods for simulating turbulent flows, 2023.
    [54] C. J. Dsilva, R. Talmon, C. W. Gear, R. R. Coifman and I. G. Kevrekidis, Data-driven reduction for a class of multiscale fast-slow stochastic dynamical systems, SIAM Journal on Applied Dynamical Systems, 15 (2016), 1327-1351.  doi: 10.1137/151004896.
    [55] Q. Du, Nonlocal modeling, Analysis and Computation, CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 94, 2020.
    [56] E. Dupont, A. Doucet and Y. W. Teh, Augmented neural ODEs, in Advances in Neural Information Processing Systems, vol. 32, Curran Associates, Inc., 2019.
    [57] K. Duraisamy, Perspectives on machine learning-augmented reynolds-averaged and large eddy simulation models of turbulence, Physical Review Fluids, 6 (2021), 050504.  doi: 10.1103/PhysRevFluids.6.050504.
    [58] K. Duraisamy, G. Iaccarino and H. Xiao, Turbulence modeling in the age of data, Annual Review of Fluid Mechanics, 51 (2019), 357-377.  doi: 10.1146/annurev-fluid-010518-040547.
    [59] A. E. Durumeric, N. E. Charron, C. Templeton, F. Musil, K. Bonneau, A. S. Pasos-Trejo, Y. Chen, A. Kelkar, F. Noé and C. Clementi, Machine learned coarse-grained protein force-fields: Are we there yet?, Current Opinion in Structural Biology, 79 (2023), 102533.  doi: 10.1016/j.sbi.2023.102533.
    [60] M. D'Elia, Q. Du, C. Glusa, M. Gunzburger, X. Tian and Z. Zhou, Numerical methods for nonlocal and fractional models, Acta Numerica, 29 (2020), 1-124.  doi: 10.1017/S096249292000001X.
    [61] W. E,  Principles of Multiscale Modeling, Cambridge University Press, Cambridge, New York, 2011. 
    [62] W. E, J. Han and L. Zhang, Integrating machine learning with physics-based modeling, 2020.
    [63] W. E, H. Lei, P. Xie and L. Zhang, Machine learning-assisted multi-scale modeling, Journal of Mathematical Physics, 64 (2023), 071101.  doi: 10.1063/5.0149861.
    [64] W. E and B. Yu, The deep ritz method: A deep learning-based numerical algorithm for solving variational problems, Communications in Mathematics and Statistics, 6 (2018), 1-12.  doi: 10.1007/s40304-018-0127-z.
    [65] W. N. Edeling, Quantification of Modelling Uncertainties in Turbulent Flow Simulations, PhD thesis, Delft University of Technology, 2015.
    [66] W. N. Edeling, P. Cinnella, R. P. Dwight and H. Bijl, Bayesian estimates of parameter variability in the k-$\varepsilon$ turbulence model, Journal of Computational Physics, 258 (2014), 73-94.  doi: 10.1016/j.jcp.2013.10.027.
    [67] H. Eivazi, M. Tahani, P. Schlatter and R. Vinuesa, Physics-informed neural networks for solving reynolds-averaged navier-stokes equations, Physics of Fluids, 34 (2022). doi: 10.1063/5.0095270.
    [68] H. Fan, J. Jiang, C. Zhang, X. Wang and Y.-C. Lai, Long-term prediction of chaotic systems with machine learning, Physical Review Research, 2 (2020), 012080.  doi: 10.1103/PhysRevResearch.2.012080.
    [69] U. Fasel, J. N. Kutz, B. W. Brunton and S. L. Brunton, Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 478 (2022), 20210904.  doi: 10.1098/rspa.2021.0904.
    [70] M. Fiore, L. Koloszar, C. Fare, M. A. Mendez, M. Duponcheel and Y. Bartosiewicz, Physics-constrained machine learning for thermal turbulence modelling at low prandtl numbers, International Journal of Heat and Mass Transfer, 194 (2022), 122998.  doi: 10.1016/j.ijheatmasstransfer.2022.122998.
    [71] H. Frezat, G. Balarac, J. L. Sommer and R. Fablet, Gradient-free online learning of subgrid-scale dynamics with neural emulators, 2023.
    [72] H. Frezat, J. Le Sommer, R. Fablet, G. Balarac and R. Lguensat, A posteriori learning for quasi-geostrophic turbulence parametrization, Journal of Advances in Modeling Earth Systems, 14 (2022), e2022MS003124. doi: 10.1029/2022MS003124.
    [73] K. Fukami, K. Fukagata and K. Taira, Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows, Journal of Fluid Mechanics, 909 (2021), A9. doi: 10.1017/jfm.2020.948.
    [74] K. Fukami, T. Murata, K. Zhang and K. Fukagata, Sparse identification of nonlinear dynamics with low-dimensionalized flow representations, Journal of Fluid Mechanics, 926 (2021), A10. doi: 10.1017/jfm.2021.697.
    [75] M. Gamahara and Y. Hattori, Searching for turbulence models by artificial neural network, Physical Review Fluids, 2 (2017), 054604.  doi: 10.1103/PhysRevFluids.2.054604.
    [76] P. Garnier, J. Viquerat, J. Rabault, A. Larcher, A. Kuhnle and E. Hachem, A review on deep reinforcement learning for fluid mechanics, Comput. Amp; Fluids 225 (2021), 13 pp. doi: 10.1016/j.compfluid.2021.104973.
    [77] C. W. Gear and I. G. Kevrekidis, Projective methods for stiff differential equations: problems with gaps in their eigenvalue spectrum, SIAM Journal on Scientific Computing, 24 (2003), 1091-1106.  doi: 10.1137/S1064827501388157.
    [78] N. Geneva and N. Zabaras, Multi-fidelity generative deep learning turbulent flows, Foundations of Data Science, 2 (2020), 391-428.  doi: 10.3934/fods.2020019.
    [79] M. Giselle Fernández-Godino, C. Park, N. H. Kim and R. T. Haftka, Issues in deciding whether to use multifidelity surrogates, AIAA Journal, 57 (2019), 2039-2054.  doi: 10.2514/1.J057750.
    [80] D. Givon, R. Kupferman and A. Stuart, Extracting macroscopic dynamics: Model problems and algorithms, Nonlinearity, 17 (2004), R55-R127. doi: 10.1088/0951-7715/17/6/R01.
    [81] E. Goan and C. Fookes, Bayesian neural networks: An introduction and survey, Case Studies in Applied Bayesian Data Science: CIRM Jean-Morlet Chair, Fall 2018, 2259 (2020), 45-87. 
    [82] F. Gonzalez, F.-X. Demoulin and S. Bernard, Towards Long-Term predictions of Turbulence using Neural Operators, 2023.
    [83] I. Goodfellow,  Y. Bengio and  A. Courville,  Deep Learning, Adaptive Computation and Machine Learning, The MIT Press, Cambridge, Massachusetts, 2016. 
    [84] F. F. Grinstein, L. G. Margolin and W. J. Rider (eds.), Implicit Large Eddy Simulation: Computing Turbulent Fluid Dynamics, Edited by Fernando F. Grinstein, Len G. Margolin and William J. Rider Cambridge University Press, Cambridge, 2007
    [85] Y. Guan, A. Chattopadhyay, A. Subel and P. Hassanzadeh, Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning, Journal of Computational Physics, 458 (2022), 111090.  doi: 10.1016/j.jcp.2022.111090.
    [86] Y. Guan, A. Subel, A. Chattopadhyay and P. Hassanzadeh, Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES, Physica D: Nonlinear Phenomena, 443 (2023), 14 pp.
    [87] J. Gullbrand and F. K. Chow, The effect of numerical errors and turbulence models in large-eddy simulations of channel flow, with and without explicit filtering, Journal of Fluid Mechanics, 495 (2003), 323-341. 
    [88] M. D. Gunzburger, Perspectives in Flow Control and Optimization, Advances in Design and Control, Society for Industrial and Applied Mathematics, Philadelphia, PA, 2003. doi: 10.1115/1.1623758.
    [89] A. Gupta and P. F. J. Lermusiaux, Neural closure models for dynamical systems, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 477 (2021), 20201004.  doi: 10.1098/rspa.2020.1004.
    [90] E. Haber, F. Lucka and L. Ruthotto, Never look back-a modified EnKF method and its application to the training of neural networks without back propagation, preprint, arXiv: 1805.08034.
    [91] J. Han, C. Ma, Z. Ma and W. E, Uniformly accurate machine learning-based hydrodynamic models for kinetic equations, Proceedings of the National Academy of Sciences, 116 (2019), 21983-21991.  doi: 10.1073/pnas.1909854116.
    [92] J. Han, X.-H. Zhou and H. Xiao, An equivariant neural operator for developing nonlocal tensorial constitutive models, Journal of Computational Physics, 488 (2023), 112243.  doi: 10.1016/j.jcp.2023.112243.
    [93] S. Hanrahan, M. Kozul and R. Sandberg, Studying turbulent flows with physics-informed neural networks and sparse data, International Journal of Heat and Fluid Flow, 104 (2023), 109232.  doi: 10.1016/j.ijheatfluidflow.2023.109232.
    [94] P. C. Hansen, Discrete Inverse Problems: Insight and Algorithms, Fundamentals of Algorithms, Society for Industrial and Applied Mathematics, Philadelphia, 2010.
    [95] D. Hartmann and L. Failer, A Differentiable Solver Approach to Operator Inference, 2021.
    [96] E. D. Held, J. D. Callen, C. C. Hegna, C. R. Sovinec, T. A. Gianakon and S. E. Kruger, Nonlocal closures for plasma fluid simulations, Physics of plasmas, 11 (2004), 2419-2426.  doi: 10.1063/1.1645520.
    [97] J. S. Hesthaven, G. Rozza and B. Stamm, Certified Reduced Basis Methods for Parametrized Partial Differential Equations, SpringerBriefs in Mathematics, Springer International Publishing, Cham, 2016.
    [98] M. Hinze, R. Pinneau, M. Ulbrich and S. Ulbrich, Optimization with PDE Constraints, Math. Model. Theory Appl., 23 Springer, New York, 2009.
    [99] J. Holland, J. Baeder and K. Duraisamy, Towards integrated field inversion and machine learning with embedded neural networks for RANS modeling, in AIAA SciTech 2019 Forum, San Diego, California, USA, 2019.
    [100] A. A. Howard, M. Perego, G. E. Karniadakis and P. Stinis, Multifidelity deep operator networks for data-driven and physics-informed problems, Journal of Computational Physics, 493 (2023), 112462.  doi: 10.1016/j.jcp.2023.112462.
    [101] J. Huang, Y. Cheng, A. J. Christlieb and L. F. Roberts, Machine learning moment closure models for the radiative transfer equation I: Directly learning a gradient based closure, Journal of Computational Physics, 453 (2022), 110941.  doi: 10.1016/j.jcp.2022.110941.
    [102] T. Hudson and X. H. Li, Coarse-graining of overdamped langevin dynamics via the Mori-Zwanzig formalism, Multiscale Modeling Simulation, 18 (2020), 1113-1135. 
    [103] K. Jakhar, Y. Guan, R. Mojgani, A. Chattopadhyay, P. Hassanzadeh and L. Zanna, Learning closed-form equations for subgrid-scale closures from high-fidelity data: Promises and challenges, 2023.
    [104] A. Javili, R. Morasata, E. Oterkus and S. Oterkus, Peridynamics review, Mathematics and Mechanics of Solids, 24 (2019), 3714-3739.  doi: 10.1177/1081286518803411.
    [105] A. S. Joglekar and A. G. Thomas, Machine learning of hidden variables in multiscale fluid simulation, arXiv preprint arXiv: 2306.10709.
    [106] L. V. Jospin, H. Laga, F. Boussaid, W. Buntine and M. Bennamoun, Hands-on Bayesian neural networks—a tutorial for deep learning users, IEEE Computational Intelligence Magazine, 17 (2022), 29-48.  doi: 10.1109/MCI.2022.3155327.
    [107] G. Jung, M. Hanke and F. Schmid, Iterative reconstruction of memory kernels, Journal of Chemical Theory and Computation, 13 (2017), 2481-2488.  doi: 10.1021/acs.jctc.7b00274.
    [108] V. Kag, K. Seshasayanan and V. Gopinath, Physics-informed data based neural networks for two-dimensional turbulence, Physics of Fluids, 34.
    [109] S. Kang and E. M. Constantinescu, Learning subgrid-scale models in discontinuous galerkin methods with neural ordinary differential equations for compressible navier-stokes equations, Comput. amp; Fluids, 261 (2023), 15 pp.
    [110] K. Kanov, R. Burns, C. Lalescu and G. Eyink, The Johns Hopkins turbulence databases: An open simulation laboratory for turbulence research, Computing in Science Engineering, 17 (2015), 10-17. 
    [111] K. Karapiperis, L. Stainier, M. Ortiz and J. E. Andrade, Data-driven multiscale modeling in mechanics, Journal of the Mechanics and Physics of Solids, 147 (2021), 104239.  doi: 10.1016/j.jmps.2020.104239.
    [112] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang and L. Yang, Physics-informed machine learning, Nature Reviews Physics, 3 (2021), 422-440.  doi: 10.1038/s42254-021-00314-5.
    [113] K. Kashinath, P. Marcus et al., Enforcing physical constraints in cnns through differentiable pde layer, in ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations, 2020.
    [114] R. Keisler, Forecasting global weather with graph neural networks, preprint, arXiv: 2202.07575.
    [115] I. G. Kevrekidis, C. W. Gear, J. M. Hyman, P. G. Kevrekidis, O. Runborg and C. Theodoropoulos, Equation-free, coarse-grained multiscale computation: Enabling microscopic simulators to perform system-level analysis, Communications in Mathematical Sciences, 1 (2003), 715-762.  doi: 10.4310/CMS.2003.v1.n4.a5.
    [116] D. E. Keyes, L. C. McInnes, C. Woodward, W. Gropp, E. Myra, M. Pernice, J. Bell, J. Brown, A. Clo, J. Connors et al., Multiphysics simulations: Challenges and opportunities, The International Journal of High Performance Computing Applications, 27 (2013), 4-83.
    [117] E. Kharazmi, Z. Mao, G. Pang, M. Zayernouri and G. E. Karniadakis, Fractional calculus and numerical methods for fractional pdes, in First Congress of Greek Mathematicians: Proceedings of the Congress Held in Athens, Walter de Gruyter GmbH & Co KG, (2020), 91.
    [118] P. Kidger, On Neural Differential Equations, 2022.
    [119] J. Kim, H. Kim, J. Kim and C. Lee, Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence, Physics of Fluids, 34 (2022), 105132.  doi: 10.1063/5.0106940.
    [120] J. Kim and C. Lee, Deep unsupervised learning of turbulence for inflow generation at various reynolds numbers, Journal of Computational Physics, 406 (2020), 109216.  doi: 10.1016/j.jcp.2019.109216.
    [121] T. Kirchdoerfer and M. Ortiz, Data-driven computational mechanics, Computer Methods in Applied Mechanics and Engineering, 304 (2016), 81-101.  doi: 10.1016/j.cma.2016.02.001.
    [122] D. E. Kirk, Optimal Control Theory: An Introduction, Dover Publications, Mineola, N. Y, 2004.
    [123] B. Koc, C. Mou, H. Liu, Z. Wang, G. Rozza and T. Iliescu, Verifiability of the data-driven variational multiscale reduced order model, Journal of Scientific Computing, 93 (2022), 54.  doi: 10.1007/s10915-022-02019-y.
    [124] D. Kochkov, J. A. Smith, A. Alieva, Q. Wang, M. P. Brenner and S. Hoyer, Machine learning-accelerated computational fluid dynamics, Proceedings of the National Academy of Sciences, 118 (2021), e2101784118. doi: 10.1073/pnas.2101784118.
    [125] N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart and A. Anandkumar, Neural Operator: Learning maps between function spaces with applications to PDEs, Journal of Machine Learning Research, 24 (2023), 1-97. 
    [126] N. B. Kovachki and A. M. Stuart, Ensemble Kalman inversion: a derivative-free technique for machine learning tasks, Inverse Problems, 35 (2019), 095005.  doi: 10.1088/1361-6420/ab1c3a.
    [127] B. Kramer, B. Peherstorfer and K. E. Willcox, Learning Nonlinear Reduced Models from Data with Operator Inference, Annual Review of Fluid Mechanics, 56 (2024), 521-548.  doi: 10.1146/annurev-fluid-121021-025220.
    [128] M. Kumar and N. Yadav, Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: A survey, Computers Mathematics with Applications, 62 (2011), 3796-3811. 
    [129] M. Kurz and A. Beck, Investigating model-data inconsistency in data-informed turbulence closure terms, 14th WCCM-ECCOMAS Congress, 2020.
    [130] M. Kurz and A. Beck, A machine learning framework for LES closure terms, ETNA - Electronic Transactions on Numerical Analysis, 56 (2022), 117-137.  doi: 10.1553/etna_vol56s117.
    [131] M. Kurz, P. Offenhäuser and A. Beck, Deep reinforcement learning for turbulence modeling in large eddy simulations, International Journal of Heat and Fluid Flow, 99 (2023), 109094.  doi: 10.1016/j.ijheatfluidflow.2022.109094.
    [132] I. E. Lagaris, A. Likas and D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE Transactions on Neural Networks, 9 (1998), 987-1000.  doi: 10.1109/72.712178.
    [133] B. Lakshminarayanan, A. Pritzel and C. Blundell, Simple and scalable predictive uncertainty estimation using deep ensembles, Advances in Neural Information Processing Systems, 30.
    [134] R. Lam, A. Sanchez-Gonzalez, M. Willson, P. Wirnsberger, M. Fortunato, F. Alet, S. Ravuri, T. Ewalds, Z. Eaton-Rosen, W. Hu, A. Merose, S. Hoyer, G. Holland, O. Vinyals, J. Stott, A. Pritzel, S. Mohamed and P. Battaglia, GraphCast: Learning skillful medium-range global weather forecasting, Science, 382 (2023), 1416-1421. 
    [135] G. Lamberti and C. Gorlé, A multi-fidelity machine learning framework to predict wind loads on buildings, Journal of Wind Engineering and Industrial Aerodynamics, 214 (2021), 104647.  doi: 10.1016/j.jweia.2021.104647.
    [136] S. Lee, Y. M. Psarellis, C. I. Siettos and I. G. Kevrekidis, Learning black-and gray-box chemotactic pdes/closures from agent based monte carlo simulation data, Journal of Mathematical Biology, 87 (2023), 15.  doi: 10.1007/s00285-023-01946-0.
    [137] H. Lei, N. Baker and X. Li, Data-driven parameterization of the generalized Langevin equation, Proceedings of the National Academy of Sciences, 113 (2016), 14183-14188.  doi: 10.1073/pnas.1609587113.
    [138] B. Leimkuhler and M. Sachs, Efficient numerical algorithms for the generalized langevin equation, SIAM Journal on Scientific Computing, 44 (2022), A364-A388. doi: 10.1137/20M138497X.
    [139] C. D. Levermore, Moment closure hierarchies for kinetic theories, Journal of Statistical Physics, 83 (1996), 1021-1065.  doi: 10.1007/BF02179552.
    [140] M. Levine and A. Stuart, A framework for machine learning of model error in dynamical systems, Communications of the American Mathematical Society, 2 (2022), 283-344.  doi: 10.1090/cams/10.
    [141] J. Li, Y. Li, T. Liu, D. Zhang and Y. Xie, Multi-fidelity graph neural network for flow field data fusion of turbomachinery, Energy, 285 (2023), 129405.  doi: 10.1016/j.energy.2023.129405.
    [142] Z. Li, X. Bian, X. Li and G. E. Karniadakis, Incorporation of memory effects in coarse-grained modeling via the Mori-Zwanzig formalism, The Journal of Chemical Physics, 143 (2015), 3128.  doi: 10.1063/1.4935490.
    [143] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart and A. Anandkumar, Fourier neural operator for parametric partial differential equations, 2021.
    [144] Z. Li, M. Liu-Schiaffini, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart and A. Anandkumar, Learning chaotic dynamics in dissipative systems, Advances in Neural Information Processing Systems, 35 (2022), 16768-16781. 
    [145] Z. Li, W. Peng, Z. Yuan and J. Wang, Fourier neural operator approach to large eddy simulation of three-dimensional turbulence, Theoretical and Applied Mechanics Letters, 12 (2022), 100389.  doi: 10.1016/j.taml.2022.100389.
    [146] P. Liao, W. Song, P. Du and H. Zhao, Multi-fidelity convolutional neural network surrogate model for aerodynamic optimization based on transfer learning, Physics of Fluids, 33.
    [147] K. K. Lin and F. Lu, Data-driven model reduction, Wiener projections, and the Koopman-Mori-Zwanzig formalism, Journal of Computational Physics, 424 (2021), 109864.  doi: 10.1016/j.jcp.2020.109864.
    [148] J. Ling, R. Jones and J. Templeton, Machine learning strategies for systems with invariance properties, Journal of Computational Physics, 318 (2016), 22-35.  doi: 10.1016/j.jcp.2016.05.003.
    [149] J. Ling, A. Kurzawski and J. Templeton, Reynolds averaged turbulence modelling using deep neural networks with embedded invariance, Journal of Fluid Mechanics, 807 (2016), 155-166.  doi: 10.1017/jfm.2016.615.
    [150] M. Lino, S. Fotiadis, A. A. Bharath and C. D. Cantwell, Current and emerging deep-learning methods for the simulation of fluid dynamics, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 479 (2023), 20230058.  doi: 10.1098/rspa.2023.0058.
    [151] P. Lippe, B. S. Veeling, P. Perdikaris, R. E. Turner and J. Brandstetter, PDE-refiner: Achieving accurate long rollouts with neural PDE solvers, 2023.
    [152] B. List, L.-W. Chen and N. Thuerey, Learned turbulence modelling with differentiable fluid solvers: Physics-based loss functions and optimisation horizons, Journal of Fluid Mechanics, 949 (2022), A25. doi: 10.1017/jfm.2022.738.
    [153] A. Liu, J. Axås and G. Haller, Data-driven modeling and forecasting of chaotic dynamics on inertial manifolds constructed as spectral submanifolds, Chaos, 34 (2024), 20 pp.
    [154] B. Liu, H. Yu, H. Huang, N. Liu and X. Lu, Investigation of nonlocal data-driven methods for subgrid-scale stress modeling in large eddy simulation, AIP Advances, 12.
    [155] J. Liu, H. H. Williams and A. Mani, Systematic approach for modeling a nonlocal eddy diffusivity, Physical Review Fluids, 8 (2023), 124501.  doi: 10.1103/PhysRevFluids.8.124501.
    [156] J.-C. Loiseau, B. R. Noack and S. L. Brunton, Sparse reduced-order modelling: sensor-based dynamics to full-state estimation, Journal of Fluid Mechanics, 844 (2018), 459-490.  doi: 10.1017/jfm.2018.147.
    [157] I. Lopez-Gomez, C. Christopoulos, H. L. Langeland Ervik, O. R. Dunbar, Y. Cohen and T. Schneider, Training physics-based machine-learning parameterizations with gradient-free ensemble kalman methods, Journal of Advances in Modeling Earth Systems, 14 (2022), e2022MS003105. doi: 10.1029/2022MS003105.
    [158] L. Lu, P. Jin, G. Pang, Z. Zhang and G. E. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence, 3 (2021), 218-229.  doi: 10.1038/s42256-021-00302-5.
    [159] T. Lund, The use of explicit filters in large eddy simulation, Computers Mathematics with Applications, 46 (2003), 603-616. 
    [160] B. Lütjens, C. H. Crawford, C. D. Watson, C. Hill and D. Newman, Multiscale Neural Operator: Learning Fast and Grid-independent PDE Solvers, 2022.
    [161] C. Ma, J. Wang et al., Model reduction with memory and the machine learning of dynamical systems, Commun. Comput. Phys., 25 (2019), 947-962.
    [162] L. Ma, X. Li and C. Liu, Coarse-graining langevin dynamics using reduced-order techniques, Journal of Computational Physics, 380 (2019), 170-190.  doi: 10.1016/j.jcp.2018.11.035.
    [163] J. F. MacArt, J. Sirignano and J. B. Freund, Embedded training of neural-network sub-grid-scale turbulence models, arXiv: 2105.01030 [physics].
    [164] S. Maddu, S. Weady and M. J. Shelley, Learning fast, accurate, and stable closures of a kinetic theory of an active fluid, J. Comput. Phys., 504 (2024), Paper No. 112869.
    [165] A. J. Majda and A. L. Bertozzi, Vorticity and Incompressible Flow, 1st edition
    [166] D. Maruyama, P. Bekemeyer, S. Görtz, S. Coggon and S. Sharma, Data-driven Bayesian inference of turbulence model closure coefficients incorporating epistemic uncertainty, Acta Mechanica Sinica, 37 (2021), 1812-1838.  doi: 10.1007/s10409-021-01152-5.
    [167] S. Massaroli, M. Poli, J. Park, A. Yamashita and H. Asama, Dissecting neural ODEs, in Advances in Neural Information Processing Systems, Curran Associates, Inc., 33 (2020), 3952-3963.
    [168] R. Maulik and O. San, A neural network approach for the blind deconvolution of turbulent flows, Journal of Fluid Mechanics, 831 (2017), 151-181.  doi: 10.1017/jfm.2017.637.
    [169] R. Maulik and O. San, A dynamic closure modeling framework for large eddy simulation using approximate deconvolution: Burgers equation, Cogent Physics, 5 (2018), 1464368.  doi: 10.1080/23311940.2018.1464368.
    [170] R. Maulik, O. San, A. Rasheed and P. Vedula, Subgrid modelling for two-dimensional turbulence using neural networks, Journal of Fluid Mechanics, 858 (2019), 122-144.  doi: 10.1017/jfm.2018.770.
    [171] H. A. Melchers, Machine Learning for Closure Models, Master's thesis, Eindhoven University of Technology, 2022.
    [172] H. Melchers, D. Crommelin, B. Koren, V. Menkovski and B. Sanderse, Comparison of neural closure models for discretised PDEs, Computers Mathematics with Applications, 143 (2023), 94-107. 
    [173] C. Meng, S. Seo, D. Cao, S. Griesemer and Y. Liu, When physics meets machine learning: A survey of physics-informed machine learning, preprint, arXiv: 2203.16797.
    [174] M. Mohebujjaman, L. Rebholz and T. Iliescu, Physically constrained data-driven correction for reduced-order modeling of fluid flows, International Journal for Numerical Methods in Fluids, 89 (2019), 103-122.  doi: 10.1002/fld.4684.
    [175] R. Mojgani, A. Chattopadhyay and P. Hassanzadeh, Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case, Chaos: An Interdisciplinary Journal of Nonlinear Science, 32 (2022), 061105.  doi: 10.1063/5.0091282.
    [176] S. Mondal and S. Sarkar, Multi-fidelity prediction of spatiotemporal fluid flow, Physics of Fluids, 34 (2022). doi: 10.1063/5.0099197.
    [177] C. Mou, N. Chen and T. Iliescu, An efficient data-driven multiscale stochastic reduced order modeling framework for complex systems, Journal of Computational Physics, 493 (2023), 112450.  doi: 10.1016/j.jcp.2023.112450.
    [178] C. Mou, B. Koc, O. San, L. G. Rebholz and T. Iliescu, Data-driven variational multiscale reduced order models, Computer Methods in Applied Mechanics and Engineering, 373 (2021), 113470.  doi: 10.1016/j.cma.2020.113470.
    [179] G. Novati, H. L. de Laroussilhe and P. Koumoutsakos, Automating turbulence modelling by multi-agent reinforcement learning, Nature Machine Intelligence, 3 (2021), 87-96.  doi: 10.1038/s42256-020-00272-0.
    [180] D. Onken and L. Ruthotto, Discretize-optimize vs. optimize-Discretize for, Time-Series Regression and Continuous Normalizing Flows, 2020.
    [181] H. A. Pahlavan, P. Hassanzadeh and M. J. Alexander, Explainable offline-online training of neural networks for parameterizations: A 1D gravity wave-QBO testbed in the small-data regime, Geophysical Research Letters, 51 (2024), e2023GL106324. doi: 10.1029/2023GL106324.
    [182] S. Pan and K. Duraisamy, Data-driven discovery of closure models, SIAM Journal on Applied Dynamical Systems, 17 (2018), 2381-2413.  doi: 10.1137/18M1177263.
    [183] E. J. Parish and K. Duraisamy, A paradigm for data-driven predictive modeling using field inversion and machine learning, Journal of computational physics, 305 (2016), 758-774.  doi: 10.1016/j.jcp.2015.11.012.
    [184] E. J. Parish and K. Duraisamy, A dynamic subgrid scale model for Large Eddy Simulations based on the Mori-Zwanzig formalism, Journal of Computational Physics, 349 (2017), 154-175.  doi: 10.1016/j.jcp.2017.07.053.
    [185] E. J. Parish and K. Duraisamy, Non-Markovian closure models for large eddy simulations using the Mori-Zwanzig Formalism, Physical Review Fluids, 2 (2017), 014604.  doi: 10.1103/PhysRevFluids.2.014604.
    [186] J. Park and H. Choi, Toward neural-network-based large eddy simulation: Application to turbulent channel flow, Journal of Fluid Mechanics, 914 (2021), A16. doi: 10.1017/jfm.2020.931.
    [187] J. Pathak, B. Hunt, M. Girvan, Z. Lu and E. Ott, Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach, Physical Review Letters, 120 (2018), 024102.  doi: 10.1103/PhysRevLett.120.024102.
    [188] J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizzadenesheli, P. Hassanzadeh, K. Kashinath and A. Anandkumar, FourCastNet: A global data-driven high-resolution weather model using adaptive fourier neural operators, 2022.
    [189] J. Pathak, A. Wikner, R. Fussell, S. Chandra, B. Hunt, M. Girvan and E. Ott, Hybrid forecasting of chaotic processes: Using machine learning in conjunction with a knowledge-based model, Chaos: An Interdisciplinary Journal of Nonlinear Science, 28 (2018), 041101.  doi: 10.1063/1.5028373.
    [190] G. A. Pavliotis and A. M. Stuart, Multiscale Methods, vol. 53 of Texts Applied in Mathematics, Springer New York, New York, NY, 2008.
    [191] S. Pawar and O. San, Data assimilation empowered neural network parametrizations for subgrid processes in geophysical flows, Physical Review Fluids, 6 (2021), 050501.  doi: 10.1103/PhysRevFluids.6.050501.
    [192] S. Pawar, O. San, A. Nair, A. Rasheed and T. Kvamsdal, Model fusion with physics-guided machine learning: Projection-based reduced-order modeling, Physics of Fluids, 33.
    [193] S. Pawar, O. San, A. Rasheed and P. Vedula, A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence, Theoretical and Computational Fluid Dynamics, 34 (2020), 429-455.  doi: 10.1007/s00162-019-00512-z.
    [194] S. Pawar, O. San, A. Rasheed and P. Vedula, Frame invariant neural network closures for Kraichnan turbulence, Physica A: Statistical Mechanics and its Applications, 609 (2023), 128327.  doi: 10.1016/j.physa.2022.128327.
    [195] C. Pedersen, L. Zanna, J. Bruna and P. Perezhogin, Reliable coarse-grained turbulent simulations through combined offline learning and neural emulation, 2023.
    [196] B. Peherstorfer and K. Willcox, Data-driven operator inference for nonintrusive projection-based model reduction, Computer Methods in Applied Mechanics and Engineering, 306 (2016), 196-215.  doi: 10.1016/j.cma.2016.03.025.
    [197] B. Peherstorfer, K. Willcox and M. Gunzburger, Survey of multifidelity methods in uncertainty propagation, inference, and optimization, SIAM Review, 60 (2018), 550-591.  doi: 10.1137/16M1082469.
    [198] T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez and P. W. Battaglia, Learning Mesh-Based Simulation with Graph Networks, 2021.
    [199] L. Pontryagin, Mathematical Theory of Optimal Processes, 1$^{st}$ edition, Routledge, 1986.
    [200] S. B. Pope, A more general effective-viscosity hypothesis, Journal of Fluid Mechanics, 72 (1975), 331-340.  doi: 10.1017/S0022112075003382.
    [201] A. Prakash, K. E. Jansen and J. A. Evans, Invariant data-driven subgrid stress modeling in the strain-rate eigenframe for large eddy simulation, Computer Methods in Applied Mechanics and Engineering, 399 (2022), 115457.  doi: 10.1016/j.cma.2022.115457.
    [202] J. Price, B. Meuris, M. Shapiro and P. Stinis, Optimal renormalization of multiscale systems, Proceedings of the National Academy of Sciences, 118 (2021), e2102266118. doi: 10.1073/pnas.2102266118.
    [203] J. Price and P. Stinis, Renormalized reduced order models with memory for long time prediction, Multiscale Modeling Simulation, 17 (2019), 68-91. 
    [204] D. Qi and J. Harlim, Machine learning-based statistical closure models for turbulent dynamical systems, Philosophical Transactions of the Royal Society A, 380 (2022), 20210205.  doi: 10.1098/rsta.2021.0205.
    [205] C. Rackauckas, Y. Ma, J. Martensen, C. Warner, K. Zubov, R. Supekar, D. Skinner, A. Ramadhan and A. Edelman, Universal differential equations for scientific machine learning, 2021.
    [206] N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio and A. Courville, On the spectral bias of neural networks, in International Conference on Machine Learning
    [207] M. Raissi, P. Perdikaris and G. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics, 378 (2019), 686-707.  doi: 10.1016/j.jcp.2018.10.045.
    [208] B. Raonic, R. Molinaro, T. De Ryck, T. Rohner, F. Bartolucci, R. Alaifari, S. Mishra and E. de Bézenac, Convolutional neural operators for robust and accurate learning of PDEs, Advances in Neural Information Processing Systems, 36.
    [209] S. Rasp, Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations: General algorithms and Lorenz 96 case study (v1.0), Geoscientific Model Development, 13 (2020), 2185-2196.  doi: 10.5194/gmd-13-2185-2020.
    [210] S. Rasp, P. D. Dueben, S. Scher, J. A. Weyn, S. Mouatadid and N. Thuerey, WeatherBench: a benchmark data set for data-driven weather forecasting, Journal of Advances in Modeling Earth Systems, 12 (2020), e2020MS002203. doi: 10.1029/2020MS002203.
    [211] S. Rasp, M. S. Pritchard and P. Gentine, Deep learning to represent subgrid processes in climate models, Proceedings of the National Academy of Sciences, 115 (2018), 9684-9689.  doi: 10.1073/pnas.1810286115.
    [212] D. Razafindralandy and A. Hamdouni, Consequences of symmetries on the analysis and construction of turbulence models, Symmetry, Integrability and Geometry: Methods and Applications, 2 (2006), 20 pp.
    [213] S. H. Rudy, S. L. Brunton, J. L. Proctor and J. N. Kutz, Data-driven discovery of partial differential equations, Science Advances, 3 (2017), e1602614. doi: 10.1126/sciadv.1602614.
    [214] A. Russo, M. A. Durán-Olivencia, I. G. Kevrekidis and S. Kalliadasis, Machine learning memory kernels as closure for non-markovian stochastic processes, IEEE Transactions on Neural Networks and Learning Systems.
    [215] P. Sagaut, Large Eddy Simulation for Incompressible Flows: An Introduction, 3rd edition, Scientific Computation, Springer, Berlin; New York, 2006.
    [216] O. San, S. Pawar and A. Rasheed, Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems, Scientific Reports, 12 (2022), 17947.  doi: 10.1038/s41598-022-22598-y.
    [217] A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec and P. Battaglia, Learning to simulate complex physics with graph networks, in Proceedings of the 37th International Conference on Machine Learning, PMLR, (2020), 8459-8468.
    [218] J. A. Sanders, F. Verhulst and J. Murdock, Averaging Methods in Nonlinear Dynamical Systems, vol. 59, Springer, 2007.
    [219] M. G. Saunders and G. A. Voth, Coarse-graining methods for computational biology, Annual Review of Biophysics, 42 (2013), 73-93.  doi: 10.1146/annurev-biophys-083012-130348.
    [220] Y. Schiff, Z. Y. Wan, J. B. Parker, S. Hoyer, V. Kuleshov, F. Sha and L. Zepeda-Núñez, DySLIM: Dynamics stable learning by invariant measure for Chaotic Systems, 2024.
    [221] M. Schmelzer, R. Dwight and P. Cinnella, Data-driven deterministic symbolic regression of nonlinear stress-strain relation for RANS turbulence modelling, in 2018 Fluid Dynamics Conference, American Institute of Aeronautics and Astronautics, Atlanta, Georgia, 2018.
    [222] M. Schmelzer, R. P. Dwight and P. Cinnella, Discovery of algebraic reynolds-stress models using sparse symbolic regression, Flow, Turbulence and Combustion, 104 (2020), 579-603.  doi: 10.1007/s10494-019-00089-x.
    [223] O. Sen, N. J. Gaul, K. Choi, G. Jacobs and H. Udaykumar, Evaluation of multifidelity surrogate modeling techniques to construct closure laws for drag in shock-particle interactions, Journal of Computational Physics, 371 (2018), 434-451.  doi: 10.1016/j.jcp.2018.05.039.
    [224] S. H. Seyedi and M. Zayernouri, A data-driven dynamic nonlocal subgrid-scale model for turbulent flows, Physics of Fluids, 34.
    [225] V. Shankar, S. Barwey, Z. Kolter, R. Maulik and V. Viswanathan, Importance of equivariant and invariant symmetries for fluid flow modeling, preprint, arXiv: 2307.05486.
    [226] V. Shankar, R. Maulik and V. Viswanathan, Differentiable turbulence, preprint, arXiv:2307.03683, 2023.
    [227] V. Shankar, R. Maulik and V. Viswanathan, Differentiable turbulence ii, preprint, arXiv: 2307.13533.
    [228] V. Shankar, V. Puri, R. Balakrishnan, R. Maulik and V. Viswanathan, Differentiable physics-enabled closure modeling for Burgers' turbulence, Machine Learning: Science and Technology, 4 (2022). doi: 10.1088/2632-2153/acb19c.
    [229] Z. She, P. Ge and H. Lei, Data-driven construction of stochastic reduced dynamics encoded with non-markovian features, The Journal of Chemical Physics, 158.
    [230] Z. Shen, A. Sridhar, Z. Tan, A. Jaruga and T. Schneider, A library of large-eddy simulations for calibrating cloud parameterizations, Authorea Preprints.
    [231] B. Siddani, S. Balachandar and R. Fang, Rotational and reflectional equivariant convolutional neural network for data-limited applications: Multiphase flow demonstration, Physics of Fluids, 33 (2021), 103323.  doi: 10.1063/5.0066049.
    [232] M. H. Silvis, R. A. Remmerswaal and R. Verstappen, Physical consistency of subgrid-scale models for large-eddy simulation of incompressible turbulent flows, Physics of Fluids, 29 (2017), 015105.  doi: 10.1063/1.4974093.
    [233] J. Sirignano and J. F. MacArt, Dynamic deep Learning LES closures: Online optimization with embedded DNS, 2023.
    [234] J. Sirignano, J. F. MacArt and J. B. Freund, DPM: A deep learning PDE augmentation method with application to large-eddy simulation, Journal of Computational Physics, 423 (2020), 109811, 21 pp. doi: 10.1016/j.jcp.2020.109811.
    [235] F. Song and G. E. Karniadakis, Variable-order fractional models for wall-bounded turbulent flows, Entropy, 23 (2021), 782.  doi: 10.3390/e23060782.
    [236] M. Sonnewald, R. Lguensat, D. C. Jones, P. D. Dueben, J. Brajard and V. Balaji, Bridging observations, theory and numerical simulation of the ocean using machine learning, Environmental Research Letters, 16 (2021), 073008.  doi: 10.1088/1748-9326/ac0eb0.
    [237] P. Spalart, An old-fashioned framework for machine learning in turbulence modeling, 2023.
    [238] C. G. Speziale, Galilean invariance of subgrid-scale stress models in the large-eddy simulation of turbulence, Journal of Fluid Mechanics, 156 (1985), 55.  doi: 10.1017/S0022112085001987.
    [239] C. G. Speziale, Turbulence Modeling in Noninertial Frames of Reference, Theoretical and Computational Fluid Dynamics, 1 (1989), 3-19.  doi: 10.1007/BF00271419.
    [240] K. Stachenfeld, D. B. Fielding, D. Kochkov, M. Cranmer, T. Pfaff, J. Godwin, C. Cui, S. Ho, P. Battaglia and A. Sanchez-Gonzalez, Learned Coarse Models for Efficient Turbulence Simulation, 2022.
    [241] P. Stinis, A phase transition approach to detecting singularities of partial differential equations, Communications in Applied Mathematics and Computational Science, 4 (2009), 217-239.  doi: 10.2140/camcos.2009.4.217.
    [242] P. Stinis, Numerical computation of solutions of the critical nonlinear Schrödinger equation after the singularity, Multiscale Modeling Simulation, 10 (2012), 48-60. 
    [243] P. Stinis, Renormalized Mori-Zwanzig-reduced models for systems without scale separation, in Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 471 (2015), 13 pp. doi: 10.1098/rspa.2014.0446.
    [244] S. Stolz and N. A. Adams, An approximate deconvolution procedure for large-eddy simulation, Physics of Fluids, 11 (1999), 1699-1701.  doi: 10.1063/1.869867.
    [245] A. M. Stuart, Inverse problems: A Bayesian perspective, Acta Numerica, 19 (2010), 451-559.  doi: 10.1017/S0962492910000061.
    [246] A. Subel, A. Chattopadhyay, Y. Guan and P. Hassanzadeh, Data-driven subgrid-scale modeling of forced burgers turbulence using deep learning with generalization to higher reynolds numbers via transfer learning, Physics of Fluids, 33.
    [247] J. L. Suzuki, M. Gulian, M. Zayernouri and M. D'Elia, Fractional modeling in action: A survey of nonlocal models for subsurface transport, turbulent flows, and anomalous materials, Journal of Peridynamics and Nonlocal modeling, 5 (2023), 392-459.  doi: 10.1007/s42102-022-00085-2.
    [248] G. Tabe Jamaat and Y. Hattori, A priori assessment of nonlocal data-driven wall modeling in large eddy simulation, Physics of Fluids, 35. doi: 10.1063/5.0146770.
    [249] S. Taghizadeh, F. D. Witherden, Y. A. Hassan and S. S. Girimaji, Turbulence closure modeling with data-driven techniques: Investigation of generalizable deep neural networks, Physics of Fluids, 33 (2021), 115132.  doi: 10.1063/5.0070890.
    [250] M. Takamoto, T. Praditia, R. Leiteritz, D. MacKinlay, F. Alesiani, D. Pflüger and M. Niepert, PDEBENCH: An Extensive Benchmark for Scientific Machine Learning, 2023.,
    [251] F. Takens, Detecting strange attractors in turbulence, in Dynamical Systems and Turbulence, Warwick 1980 (eds. D. Rand and L.-S. Young), Springer Berlin Heidelberg, Berlin, Heidelberg, 898 (1981), 366-381.
    [252] A. Towne, S. T. Dawson, G. A. Brès, A. Lozano-Durán, T. Saxton-Fox, A. Parthasarathy, A. R. Jones, H. Biler, C.-A. Yeh, H. D. Patel et al., A database for reduced-complexity modeling of fluid flows, AIAA Journal, 1-26.
    [253] B. Tzen and M. Raginsky, Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit, 2019.
    [254] K. Um, R. Brand, Y. R. Fei, P. Holl and N. Thuerey, Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers, in Advances in Neural Information Processing Systems, Curran Associates, Inc., 33 (2020), 6111-6122.
    [255] W. I. T. Uy, D. Hartmann and B. Peherstorfer, Operator inference with roll outs for learning reduced models from scarce and low-quality data, Computers Mathematics with Applications, 145 (2023), 224-239. 
    [256] H. Vaddireddy, A. Rasheed, A. E. Staples and O. San, Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data, Physics of Fluids, 32 (2020). doi: 10.1063/1.5136351.
    [257] E. van der Giessen, P. A. Schultz, N. Bertin, V. V. Bulatov, W. Cai, G. Csányi, S. M. Foiles, M. G. D. Geers, C. González, M. Hütter, W. K. Kim, D. M. Kochmann, J. LLorca, A. E. Mattsson, J. Rottler, A. Shluger, R. B. Sills, I. Steinbach, A. Strachan and E. B. Tadmor, Roadmap on multiscale materials modeling, Modelling and Simulation in Materials Science and Engineering, 28 (2020), 043001.  doi: 10.1088/1361-651X/ab7150.
    [258] T. van Gastelen, W. Edeling and B. Sanderse, Energy-conserving neural network for turbulence closure modeling, J. Comput. Phys., 508 (2024), 26 pp. doi: 10.1016/j.jcp.2024.113003.
    [259] J. Viquerat, P. Meliga, A. Larcher and E. Hachem, A review on deep reinforcement learning for fluid mechanics: An update, Physics of Fluids, 34 (2022), 111301.  doi: 10.1063/5.0128446.
    [260] P. R. Vlachas, G. Arampatzis, C. Uhler and P. Koumoutsakos, Multiscale simulations of complex systems by learning their effective dynamics, Nature Machine Intelligence, 4 (2022), 359-366.  doi: 10.1038/s42256-022-00464-w.
    [261] P. R. Vlachas, W. Byeon, Z. Y. Wan, T. P. Sapsis and P. Koumoutsakos, Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 474 (2018), 20170844.  doi: 10.1098/rspa.2017.0844.
    [262] P. R. Vlachas, J. Pathak, B. R. Hunt, T. P. Sapsis, M. Girvan, E. Ott and P. Koumoutsakos, Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics, Neural Networks, 126 (2020), 191-217.  doi: 10.1016/j.neunet.2020.02.016.
    [263] C. J. Wang and P. Golland, Discretization Invariant Learning on Neural Fields, 2022.
    [264] J. Wang, S. Olsson, C. Wehmeyer, A. Pérez, N. E. Charron, G. De Fabritiis, F. Noé and C. Clementi, Machine learning of coarse-grained molecular dynamics force fields, ACS Central Science, 5 (2019), 755-767.  doi: 10.1021/acscentsci.8b00913.
    [265] J.-X. Wang, J. Wu, J. Ling, G. Iaccarino and H. Xiao, A comprehensive physics-informed machine learning framework for predictive turbulence modeling, preprint, arXiv: 1701.07102.
    [266] J.-X. Wang, J.-L. Wu and H. Xiao, A Physics Informed Machine Learning Approach for Reconstructing Reynolds Stress Modeling Discrepancies Based on DNS Data, Physical Review Fluids, 2 (2017), 034603.  doi: 10.1103/PhysRevFluids.2.034603.
    [267] Q. Wang, N. Ripamonti and J. S. Hesthaven, Recurrent neural network closure of parametric pOD-Galerkin reduced-order models based on the Mori-Zwanzig formalism, Journal of Computational Physics, 410 (2020), 109402.  doi: 10.1016/j.jcp.2020.109402.
    [268] R. Wang, K. Kashinath, M. Mustafa, A. Albert and R. Yu, Towards physics-informed deep learning for turbulent flow prediction, in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, KDD '20, Association for Computing Machinery, New York, NY, USA, (2020), 1457-1466.
    [269] R. Wang, R. Walters and R. Yu, Incorporating symmetry into deep dynamics models for improved generalization, 2021.
    [270] S. Wiewel, M. Becher and N. Thuerey, Latent space physics: Towards learning the temporal evolution of fluid flow, in Computer graphics forum, Wiley Online Library, 38 (2019), 71-82. doi: 10.1111/cgf.13620.
    [271] H. Xiao, J.-L. Wu, J.-X. Wang, R. Sun and C. Roy, Quantifying and reducing model-form uncertainties in Reynolds-averaged Navier-Stokes simulations: A data-driven, physics-informed Bayesian approach, Journal of Computational Physics, 324 (2016), 115-136.  doi: 10.1016/j.jcp.2016.07.038.
    [272] M. Xu, J. Han, A. Lou, J. Kossaifi, A. Ramanathan, K. Azizzadenesheli, J. Leskovec, S. Ermon and A. Anandkumar, Equivariant graph neural operator for modeling 3d dynamics, preprint, arXiv: 2401.11037.
    [273] X. I. A. Yang, S. Zafar, J.-X. Wang and H. Xiao, Predictive large-eddy-simulation wall modeling via physics-informed neural networks, Physical Review Fluids, 4 (2019), 034602.  doi: 10.1103/PhysRevFluids.4.034602.
    [274] H. You, Y. Yu, N. Trask, M. Gulian and M. D'Elia, Data-driven learning of nonlocal physics from high-fidelity synthetic data, Computer Methods in Applied Mechanics and Engineering, 374 (2021), 113553.  doi: 10.1016/j.cma.2020.113553.
    [275] S. Yu, W. Hannah, L. Peng, J. Lin, M. A. Bhouri, R. Gupta, B. Lütjens, J. C. Will, G. Behrens, J. Busecke et al., ClimSim: A large multi-scale dataset for hybrid physics-ML climate emulation, Advances in Neural Information Processing Systems, 36.
    [276] Z. Yuan, C. Xie and J. Wang, Deconvolutional artificial neural network models for large eddy simulation of turbulence, Physics of Fluids, 32 (2020), 115106. 
    [277] Z. Yuan, Y. Wang, C. Xie and J. Wang, Dynamic iterative approximate deconvolution models for large-eddy simulation of turbulence, Physics of Fluids, 33 (2021), 085125.  doi: 10.1063/5.0059643.
    [278] J. Yuval and P. A. O'Gorman, Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions, Nature Communications, 11 (2020), 3295.  doi: 10.1038/s41467-020-17142-3.
    [279] L. Zanna and T. Bolton, Data-driven equation discovery of ocean mesoscale closures, Geophysical Research Letters, 47 (2020), e2020GL088376. doi: 10.1029/2020GL088376.
    [280] H. Zhang and P. Qiao, Virtual crack closure technique in peridynamic theory, Computer Methods in Applied Mechanics and Engineering, 372 (2020), 113318.  doi: 10.1016/j.cma.2020.113318.
    [281] X. Zhang, F. Xie, T. Ji, Z. Zhu and Y. Zheng, Multi-fidelity deep neural network surrogate model for aerodynamic shape optimization, Computer Methods in Applied Mechanics and Engineering, 373 (2021), 113485.  doi: 10.1016/j.cma.2020.113485.
    [282] H. Zhao, R. T. D. Combes, K. Zhang and G. Gordon, On learning invariant representations for domain adaptation, in Proceedings of the 36th International Conference on Machine Learning (eds. K. Chaudhuri and R. Salakhutdinov),Proceedings of Machine Learning Research, 97 (2019), 7523-7532, https://proceedings.mlr.press/v97/zhao19a.html.
    [283] J. Zhao and Q. Li, Mitigating distribution shift in machine learning-augmented hybrid simulation, preprint, arXiv: 2401.09259.
    [284] L. Zhao, Z. Li, B. Caswell, J. Ouyang and G. E. Karniadakis, Active learning of constitutive relation from mesoscopic dynamics for macroscopic modeling of non-Newtonian flows, Journal of Computational Physics, 363 (2018), 116-127.  doi: 10.1016/j.jcp.2018.02.039.
    [285] X.-H. Zhou, J. Han and H. Xiao, Learning nonlocal constitutive models with neural networks, Computer Methods in Applied Mechanics and Engineering, 384 (2021), 113927.  doi: 10.1016/j.cma.2021.113927.
    [286] Y. Zhu, Y.-H. Tang and C. Kim, Learning stochastic dynamics with statistics-informed neural network, Journal of Computational Physics, 474 (2023), 111819.  doi: 10.1016/j.jcp.2022.111819.
    [287] Y. Zhu and D. Venturi, Hypoellipticity and the Mori-Zwanzig formulation of stochastic differential equations, Journal of Mathematical Physics, 62 (2021), 20 pp.
    [288] R. Zwanzig,  Nonequilibrium Statistical Mechanics, Oxford University Press, 2001. 
    • Access History
  • 加载中

Figures(3)

PDF
XML
Export Citation
SHARE
Email to a Friend

Article Metrics

HTML views(711) PDF downloads(196) Cited by(0)

Access History

Other Articles By Authors

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return
    Site map Copyright © 2025 American Institute of Mathematical Sciences