American Institute of Mathematical Sciences

Advanced
doi: 10.3934/fods.2020012

Spectral methods to study the robustness of residual neural networks with infinite layers

Torsten Trimborn , , Stephan Gerster and  Giuseppe Visconti

IGPM, RWTH Aachen University, Germany

* Corresponding author: Torsten Trimborn

Published  August 2020

Fund Project: This research is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy - EXC-2023 Internet of Production - 390621612 and supported also by DFG HE5386/18, 19 and DFG 320021702/GRK2326. T. Trimborn acknowledge the support by the ERS Prep Fund - Simulation and Data Science

Recently, neural networks (NN) with an infinite number of layers have been introduced. Especially for these very large NN the training procedure is very expensive. Hence, there is interest to study their robustness with respect to input data to avoid unnecessarily retraining the network.

Typically, model-based statistical inference methods, e.g. Bayesian neural networks, are used to quantify uncertainties. Here, we consider a special class of residual neural networks and we study the case, when the number of layers can be arbitrarily large. Then, kinetic theory allows to interpret the network as a dynamical system, described by a partial differential equation. We study the robustness of the mean-field neural network with respect to perturbations in initial data by applying UQ approaches on the loss functions.

Keywords: Neural networks, kinetic equations, uncertainty quantification, stochastic Galerkin, machine learning application.
Mathematics Subject Classification: Primary: 68T07, 68T05; Secondary: 35R60.
Citation: Torsten Trimborn, Stephan Gerster, Giuseppe Visconti. Spectral methods to study the robustness of residual neural networks with infinite layers. Foundations of Data Science, doi: 10.3934/fods.2020012
References:
[1]

R. Abgrall and S. Mishra, Uncertainty quantification for hyperbolic systems of conservation laws, Handbook of numerical methods for hyperbolic problems, Handb. Numer. Anal., Elsevier/North-Holland, Amsterdam, 18 (2017), 507–544.  Google Scholar

[2]

D. Anderson and G. McNeill, Artificial neural networks technology, Kaman Sciences Corporation, 258 (1992), 1-83.   Google Scholar

[3]

T. Auld, A. W. Moore and S. F. Gull, Bayesian neural networks for internet traffic classification, IEEE Transactions on Neural Networks, 18 (2007), 223-239.   Google Scholar

[4]

G. Bastin and J.-M. Coron, Stability and Boundary Stabilization of 1-d Hyperbolic Systems, 1st edition, Progress in nonlinear differential equations and their applications, Birkhäuser, Switzerland, 2016. doi: 10.1007/978-3-319-32062-5.  Google Scholar

[5]

H. D. Beale, H. B. Demuth and M. Hagan, Neural network design, PWS, Boston. Google Scholar

[6]

M. A. Beaumont, Approximate bayesian computation, Annu. Rev. Stat. Appl., 6 (2019), 379-403.  doi: 10.1146/annurev-statistics-030718-105212.  Google Scholar

[7]

M. G. B. Blum, Approximate bayesian computation: A nonparametric perspective, J. Amer. Statist. Assoc., 105 (2010), 1178-1187.  doi: 10.1198/jasa.2010.tm09448.  Google Scholar

[8]

C. Blundell, J. J. Cornebise, K. Kavukcuoglu and D. Wierstra, Weight uncertainty in neural networks, in ICML, 2015. Google Scholar

[9]

R. H. Cameron and W. T. Martin, The orthogonal development of non-linear functionals in series of Fourier-Hermite functionals, Ann. of Math., 48 (1947), 385-392.  doi: 10.2307/1969178.  Google Scholar

[10]

J. A. Carrillo, L. Pareschi and M. Zanella, Particle based gPC methods for mean-field models of swarming with uncertainty, Commun. Comput. Phys., 25 (2019), 508-531.  doi: 10.4208/cicp.oa-2017-0244.  Google Scholar

[11]

J. A. Carrillo and M. Zanella, Monte Carlo gPC methods for diffusive kinetic flocking models with uncertainties, Vietnam J. Math., 47 (2019), 931-954.  doi: 10.1007/s10013-019-00374-2.  Google Scholar

[12]

R. T. Q. Chen, Y. Rubanova and J. B. D. K. D. Duvenaud, Neural ordinary differential equations, in Advances in Neural Information Processing Systems, Curran Associates, Inc., 31 (2018), 6571–6583. Google Scholar

[13]

R. Courant and D. Hilbert, Methods of Mathematical Physics, Vol. Ⅱ. Partial differential equations. Reprint of the 1962 original. Wiley Classics Library. A Wiley-Interscience Publication. John Wiley & Sons, Inc., New York, 1989.  Google Scholar

[14]

I. Cravero, G. Puppo, M. Semplice and G. Visconti, Cool WENO schemes, Comput. & Fluids, 169 (2018), 71-86.  doi: 10.1016/j.compfluid.2017.07.022.  Google Scholar

[15]

I. Cravero, G. Puppo, M. Semplice and G. Visconti, CWENO: Uniformly accurate reconstructions for balance laws, Math. Comp., 87 (2018), 1689-1719.  doi: 10.1090/mcom/3273.  Google Scholar

[16]

I. Cravero, M. Semplice and G. Visconti, Optimal definition of the nonlinear weights in multidimensional central WENOZ reconstructions, SIAM J. Numer. Anal., 57 (2019), 2328-2358.  doi: 10.1137/18M1228232.  Google Scholar

[17]

R. Crisovan, D. Torlo, R. Abgrall and S. Tokareva, Model order reduction for parametrized nonlinear hyperbolic problems as an application to uncertainty quantification, J. Comput. Appl. Math., 348 (2019), 466-489.  doi: 10.1016/j.cam.2018.09.018.  Google Scholar

[18]

Y. Deng, Y. Shen, K. Chen and H. Jin, Training recurrent neural network through moment matching for nlp applications, (2018), 3353–3357. Google Scholar

[19]

B. Després, G. Poëtte and D. Lucor, Robust uncertainty propagation in systems of conservation laws with the entropy closure method, Uncertainty Quantification in Computational Fluid Dynamics, Lect. Notes Comput. Sci. Eng., Springer, Heidelberg, 92 (2013), 105–149. doi: 10.1007/978-3-319-00885-1_3.  Google Scholar

[20]

D. Funaro, Polynomial Approximation of Differential Equations, Lecture Notes in Physics. New Series m: Monographs, 8. Springer-Verlag, Berlin, 1992.  Google Scholar

[21]

S. Gerster and M. Herty, Entropies and symmetrization of hyperbolic stochastic Galerkin formulations, Communications in Computational Physics, 27 (2020), 639-671.   Google Scholar

[22]

S. Gerster, M. Herty and A. Sikstel, Hyperbolic stochastic Galerkin formulation for the $p$-system, J. Comput. Phys., 395 (2019), 186-204.  doi: 10.1016/j.jcp.2019.05.049.  Google Scholar

[23]

S. Gerster, M. Herty and H. Yu, Hypocoercivity of stochastic Galerkin formulations for stabilization of kinetic equations, RWTH, preprint, 1–21. Google Scholar

[24]

F. S. Gharehchopogh, Neural networks application in software cost estimation: A case study, in 2011 International Symposium on Innovations in Intelligent Systems and Applications, IEEE, (2011), 69–73. doi: 10.1109/INISTA.2011.5946160.  Google Scholar

[25]

M. B. Giles, Multilevel Monte Carlo methods, Springer Proceedings in Mathematics & Statistics, Monte Carlo and Quasi-Monte Carlo Methods, 65 (2012), 83–103. doi: 10.1007/978-3-642-41095-6_4.  Google Scholar

[26]

P. Glasserman, Monte Carlo Methods in Financial Engineering, Springer, New York, 2004.  Google Scholar

[27]

F. Golse, On the dynamics of large particle systems in the mean field limit, in Macroscopic and Large Scale Phenomena: Coarse Graining, Mean Field Limits and Ergodicity, Lect. Notes Appl. Math. Mech., Springer, 3 (2016), 1–144. doi: 10.1007/978-3-319-26883-5_1.  Google Scholar

[28]

D. Gottlieb and D. Xiu, Galerkin method for wave equations with uncertain coefficients, Commun. Comput. Phys., 3 (2008), 505-518.   Google Scholar

[29]

A. Graves, Practical variational inference for neural networks, NIPS, (2011), 2348–2356. Google Scholar

[30]

E. Haber, F. Lucka and L. Ruthotto, Never look back - A modified EnKF method and its application to the training of neural networks without back propagation, 2018, Preprint, arXiv: 1805.08034. Google Scholar

[31]

M. H. Hassoun et al., Fundamentals of Artificial Neural Networks, MIT press, 1995. Google Scholar

[32]

K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778. doi: 10.1109/CVPR.2016.90.  Google Scholar

[33]

D. O. Hebb, The Organization of Behavior: A Neuropsychological Theory, Psychology Press, 2005. doi: 10.4324/9781410612403.  Google Scholar

[34]

M. Herty, T. Trimborn and G. Visconti, Kinetic theory for residual neural networks, arXiv preprint, arXiv: 2001.04294. Google Scholar

[35]

G. Hu, R. Li and T. Tang, A robust WENO type finite volume solver for steady Euler equations on unstructured grids, Commun. Comput. Phys., 9 (2011), 627-648.  doi: 10.4208/cicp.031109.080410s.  Google Scholar

[36]

J. Hu and S. Jin, A stochastic Galerkin method for the Boltzmann equation with uncertainty, J. Comput. Phys., 315 (2016), 150-168.  doi: 10.1016/j.jcp.2016.03.047.  Google Scholar

[37]

P.-E. Jabin, A review of the mean field limits for vlasov equations, Kinetic & Related Models, 7 (2014), 661-711.  doi: 10.3934/krm.2014.7.661.  Google Scholar

[38]

K. Janocha and W. M. Czarnecki, On loss functions for deep neural networks in classifications, 2017, Preprint, arXiv: 1702.05659v1. Google Scholar

[39]

B. Jiang, T.-Y. H Wu, C. Zheng and W. Wong, Learning summary statistic for approximate bayesian computation via deep neural network, Statistica Sinica, 27 (2017), 1595-1618.   Google Scholar

[40]

G.-S. Jiang and C.-W. Shu, Efficient implementation of weighted ENO schemes, J. Comput. Phys., 126 (1996), 202-228.  doi: 10.1006/jcph.1996.0130.  Google Scholar

[41]

S. Jin and Y. Zhu, Hypocoercivity and uniform regularity for the Vlasov-Poisson-Fokker-Planck system with uncertainty and multiple scales, SIAM J. Math. Anal., 50 (2018), 1790-1816.  doi: 10.1137/17M1123845.  Google Scholar

[42]

M. I. Jordan and T. M. Mitchell, Machine learning: Trends, perspectives, and prospects, Science, 349 (2015), 255-260.  doi: 10.1126/science.aaa8415.  Google Scholar

[43]

A. V. Joshi, Machine learning and artificial intelligence, Springer, Cham, 2020. doi: 10.1007/978-3-030-26622-6.  Google Scholar

[44]

H. M. D. Kabir, A. Khosravi, M. A. Hosen and S. Nahavandi, Neural network-based uncertainty quantification: A survey of methodologies and applications, IEEE Access, 6 (2018), 36218-36234.  doi: 10.1109/ACCESS.2018.2836917.  Google Scholar

[45]

C. Klingenberg, G. Puppo and M. Semplice, Arbitrary order finite volume well-balanced schemes for the Euler equations with gravity, SIAM J. Sci. Comput., 41 (2019), A695–A721. doi: 10.1137/18M1196704.  Google Scholar

[46]

N. B. Kovachki and A. M. Stuart, Ensemble Kalman inversion: A derivative-free technique for machine learning tasks, Inverse Probl., 35 (2019), 095005, 35 pp. doi: 10.1088/1361-6420/ab1c3a.  Google Scholar

[47]

J. Kusch and M. Frank, Intrusive methods in uncertainty quantification and their connection to kinetic theory, Int. J. Adv. Eng. Sci. Appl. Math., 10 (2018), 54-69.  doi: 10.1007/s12572-018-0211-3.  Google Scholar

[48]

O. P. Le Maître and O. M. Knio, Spectral Methods for Uncertainty Quantification, With applications to computational fluid dynamics. Scientific Computation. Springer, New York, 2010. doi: 10.1007/978-90-481-3520-2.  Google Scholar

[49]

C. Li and S. Mahadevan, Efficient approximate inference in Bayesian networks with continuous variables, Reliability Engineering & System Safety, 169 (2018), 269-280.  doi: 10.1016/j.ress.2017.08.017.  Google Scholar

[50]

C.-L. Li, W.-C. Chang, Y. Cheng, Y. Yang and B. Póczos, Mmd gan: Towards deeper understanding of moment matching network, in Advances in Neural Information Processing Systems, (2017), 2203–2213. Google Scholar

[51]

H. Lin and S. Jegelka, Resnet with one-neuron hidden layers is a universal approximator, in Advances in Neural Information Processing Systems (eds. S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett), Curran Associates, Inc., 31 (2018), 6169–6178. Google Scholar

[52]

S. Mandt, M. Hoffman and D. Blei, Stochastic gradient descent as approximate Bayesian inference, J. Mach. Learn. Res., 18 (2017), Paper No. 134, 35 pp.  Google Scholar

[53]

W. S. McCulloch and W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bull. Math. Biophys., 5 (1943), 115-133.  doi: 10.1007/BF02478259.  Google Scholar

[54]

S. Mishra, Ch. Schwab and J. Šukys, Multi-level Monte Carlo finite volume methods for nonlinear systems of conservation laws in multi-dimensions, J. Comput. Phys., 231 (2012), 3365-3388.  doi: 10.1016/j.jcp.2012.01.011.  Google Scholar

[55]

S. Mishra, Ch. Schwab and J. Šukys, Multilevel Monte Carlo finite volume methods for shallow water equations with uncertain topography in multi-dimensions, SIAM J. Sci. Comput., 34 (2012), B761–B784. doi: 10.1137/110857295.  Google Scholar

[56]

O. Nikodym, Sur une généralisation des intégrales de M. J. Radon, Fundamenta Mathematicae, 15 (1930), 131-179.  doi: 10.4064/fm-15-1-131-179.  Google Scholar

[57]

S. C. Onar, A. Ustundag, Ç. Kadaifci and B. Oztaysi, The changing role of engineering education in industry 4.0 era, in Industry 4.0: Managing The Digital Transformation, Springer, (2018), 137–151. Google Scholar

[58]

P. Pettersson, G. Iaccarino and J. Nordström, A stochastic Galerkin method for the Euler equations with Roe variable transformation, Journal of Computational Physics, 257 (2014), 481-500.  doi: 10.1016/j.jcp.2013.10.011.  Google Scholar

[59]

G. Poëtte, B. Després and D. Lucor, Uncertainty quantification for systems of conservation laws, J. Comput. Phys., 228 (2009), 2443-2467.  doi: 10.1016/j.jcp.2008.12.018.  Google Scholar

[60]

R. Pulch and E. Maten, Stochastic galerkin methods and model order reduction for linear dynamical systems, International Journal for Uncertainty Quantification, 5 (2015), 255-273.  doi: 10.1615/Int.J.UncertaintyQuantification.2015010171.  Google Scholar

[61]

R. Pulch and D. Xiu, Generalised polynomial chaos for a class of linear conservation laws, J. Sci. Comput., 51 (2012), 293-312.  doi: 10.1007/s10915-011-9511-5.  Google Scholar

[62]

D. Ray and J. S. Hesthaven, An artificial neural network as a troubled-cell indicator, J. Comput. Phys., 367 (2018), 166-191.  doi: 10.1016/j.jcp.2018.04.029.  Google Scholar

[63]

D. Ray and J. S. Hesthaven, Detecting troubled-cells on two-dimensional unstructured grids using a neural network, J. Comput. Phys., 397 (2019), 108845, 31 pp. doi: 10.1016/j.jcp.2019.07.043.  Google Scholar

[64]

N. H. Risebro, C. Schwab and F. Weber, Multilevel Monte Carlo front-tracking for random scalar conservation laws, BIT, 56 (2016), 263-292.  doi: 10.1007/s10543-015-0550-4.  Google Scholar

[65]

L. Ruthotto, S. Osher, W. Li, L. Nurbekyan and S. W. Fung, A machine learning framework for solving high-dimensional mean field game and mean field control problems, Proceedings of the National Academy of Sciences, 117, (2020), 9183–9193. Google Scholar

[66]

L. Ruthotto and E. Haber, Deep neural networks motivated by partial differential equations, J. Math. Imaging Vision, 62 (2020), 352-364.  doi: 10.1007/s10851-019-00903-1.  Google Scholar

[67]

G. Scarciotti and A. Astolfi, Data-driven model reduction by moment matching for linear and nonlinear systems, Automatica J. IFAC, 79 (2017), 340-351.  doi: 10.1016/j.automatica.2017.01.014.  Google Scholar

[68]

R. Schmitt and G. Schuh, Advances in Production Research: Proceedings of the 8th Congress of the German Academic Association for Production Technology (WGP), Aachen, November 19-20, 2018, Springer, 2018. doi: 10.1007/978-3-030-03451-1.  Google Scholar

[69]

C. Schwab and S. Tokareva, High order approximation of probabilistic shock profiles in hyperbolic conservation laws with uncertain initial data, ESAIM Math. Model. Numer. Anal., 47 (2013), 807-835.  doi: 10.1051/m2an/2012060.  Google Scholar

[70]

M. Semplice, A. Coco and G. Russo, Adaptive mesh refinement for hyperbolic systems based on third-order compact WENO reconstruction, J. Sci. Comput., 66 (2016), 692-724.  doi: 10.1007/s10915-015-0038-z.  Google Scholar

[71]

R. Shu, J. Hu and S. Jin, A stochastic Galerkin method for the Boltzmann equation with multi-dimensional random inputs using sparse wavelet bases, Numer. Math. Theory Methods Appl., 10 (2017), 465-488.  doi: 10.4208/nmtma.2017.s12.  Google Scholar

[72]

R. C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2013. Google Scholar

[73]

R. C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, Computational Science & Engineering, 12. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2014.  Google Scholar

[74]

R. J. Solomonoff, Machine Learning – Past and Future, The Dartmouth Artificial Intelligence Conference, 2006. Google Scholar

[75]

J. Stoer and R. Bulirsch, Introduction to Numerical Analysis, 3rd edition, Texts in Applied Mathematics, 12. Springer-Verlag New York, 2002. doi: 10.1007/978-0-387-21738-3.  Google Scholar

[76]

J. Šukys, S. Mishra and C. Schwab, Multi-level Monte Carlo finite difference and finite volume methods for stochastic linear hyperbolic systems, Springer Proceedings in Mathematics and Statistics, 65 (2013), 649-666.  doi: 10.1007/978-3-642-41095-6_34.  Google Scholar

[77]

T. Tang and T. Zhou, Convergence analysis for stochastic collocation methods to scalar hyperbolic equations with a random wave speed, Commun. Comput. Phys., 8 (2010), 226-248.  doi: 10.4208/cicp.060109.130110a.  Google Scholar

[78]

H. Tercan, T. Al Khawli, U. Eppelt, C. Büscher, T. Meisen and S. Jeschke, Improving the laser cutting process design by machine learning techniques, Production Engineering, 11 (2017), 195-203.  doi: 10.1007/s11740-017-0718-7.  Google Scholar

[79]

H. H. Thodberg, A review of bayesian neural networks with an application to near infrared spectroscopy, IEEE Transactions on Neural Networks, 7 (1996), 56-72.   Google Scholar

[80]

S. Tokareva, C. Schwab and S. Mishra, High order SFV and mixed SDG/FV methods for the uncertainty quantification in multidimensional conservation laws, in High Order Nonlinear Numerical Schemes for Evolutionary PDEs, Lect. Notes Comput. Sci. Eng., Springer, Cham, 99 (2014), 109–133.  Google Scholar

[81]

R. K. Tripathy and I. Bilionis, Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification, J. Comput. Phys., 375 (2018), 565-588.  doi: 10.1016/j.jcp.2018.08.036.  Google Scholar

[82]

V. Vanhoucke, A. Senior and M. Z. Mao, Improving the Speed of Neural Networks on CPUs., Deep Learning and Unsupervised Feature Learning Workshop, 2011. Google Scholar

[83]

Q. Wang, J. S. Hesthaven and D. Ray, Non-intrusive reduced order modelling of unsteady flows using artificial neural networks with application to a combustion problem, J. Comput. Phys., 384 (2019), 289-307.  doi: 10.1016/j.jcp.2019.01.031.  Google Scholar

[84]

K. Watanabe and S. G. Tzafestas, Learning algorithms for neural networks with the Kalman filters, J. Intell. Robot. Syst., 3 (1990), 305-319.  doi: 10.1007/BF00439421.  Google Scholar

[85]

N. Wiener, The homogeneous chaos, Amer. J. Math., 60 (1938), 897-936.  doi: 10.2307/2371268.  Google Scholar

[86]

D. Xiu, Numerical methods for stochastic computations, A spectral method approach. Princeton University Press, Princeton, 2010.  Google Scholar

[87]

D. Xiu and G. E. Karniadakis, The Wiener-Askey polynomial chaos for stochastic differential equations, SIAM J. Sci. Comput., 24 (2002), 619-644.  doi: 10.1137/S1064827501387826.  Google Scholar

[88]

M. Zanella, Structure preserving stochastic Galerkin methods for Fokker-Planck equations with background interactions, Math. Comput. Simulation, 168 (2020), 28-47.  doi: 10.1016/j.matcom.2019.07.012.  Google Scholar

[89]

Y. Zhu and S. Jin, The Vlasov-Poisson-Fokker-Planck system with uncertainty and a one-dimensional asymptotic preserving method, Multiscale Model. Simul., 15 (2017), 1502-1529.  doi: 10.1137/16M1090028.  Google Scholar

show all references

References:
[1]

R. Abgrall and S. Mishra, Uncertainty quantification for hyperbolic systems of conservation laws, Handbook of numerical methods for hyperbolic problems, Handb. Numer. Anal., Elsevier/North-Holland, Amsterdam, 18 (2017), 507–544.  Google Scholar

[2]

D. Anderson and G. McNeill, Artificial neural networks technology, Kaman Sciences Corporation, 258 (1992), 1-83.   Google Scholar

[3]

T. Auld, A. W. Moore and S. F. Gull, Bayesian neural networks for internet traffic classification, IEEE Transactions on Neural Networks, 18 (2007), 223-239.   Google Scholar

[4]

G. Bastin and J.-M. Coron, Stability and Boundary Stabilization of 1-d Hyperbolic Systems, 1st edition, Progress in nonlinear differential equations and their applications, Birkhäuser, Switzerland, 2016. doi: 10.1007/978-3-319-32062-5.  Google Scholar

[5]

H. D. Beale, H. B. Demuth and M. Hagan, Neural network design, PWS, Boston. Google Scholar

[6]

M. A. Beaumont, Approximate bayesian computation, Annu. Rev. Stat. Appl., 6 (2019), 379-403.  doi: 10.1146/annurev-statistics-030718-105212.  Google Scholar

[7]

M. G. B. Blum, Approximate bayesian computation: A nonparametric perspective, J. Amer. Statist. Assoc., 105 (2010), 1178-1187.  doi: 10.1198/jasa.2010.tm09448.  Google Scholar

[8]

C. Blundell, J. J. Cornebise, K. Kavukcuoglu and D. Wierstra, Weight uncertainty in neural networks, in ICML, 2015. Google Scholar

[9]

R. H. Cameron and W. T. Martin, The orthogonal development of non-linear functionals in series of Fourier-Hermite functionals, Ann. of Math., 48 (1947), 385-392.  doi: 10.2307/1969178.  Google Scholar

[10]

J. A. Carrillo, L. Pareschi and M. Zanella, Particle based gPC methods for mean-field models of swarming with uncertainty, Commun. Comput. Phys., 25 (2019), 508-531.  doi: 10.4208/cicp.oa-2017-0244.  Google Scholar

[11]

J. A. Carrillo and M. Zanella, Monte Carlo gPC methods for diffusive kinetic flocking models with uncertainties, Vietnam J. Math., 47 (2019), 931-954.  doi: 10.1007/s10013-019-00374-2.  Google Scholar

[12]

R. T. Q. Chen, Y. Rubanova and J. B. D. K. D. Duvenaud, Neural ordinary differential equations, in Advances in Neural Information Processing Systems, Curran Associates, Inc., 31 (2018), 6571–6583. Google Scholar

[13]

R. Courant and D. Hilbert, Methods of Mathematical Physics, Vol. Ⅱ. Partial differential equations. Reprint of the 1962 original. Wiley Classics Library. A Wiley-Interscience Publication. John Wiley & Sons, Inc., New York, 1989.  Google Scholar

[14]

I. Cravero, G. Puppo, M. Semplice and G. Visconti, Cool WENO schemes, Comput. & Fluids, 169 (2018), 71-86.  doi: 10.1016/j.compfluid.2017.07.022.  Google Scholar

[15]

I. Cravero, G. Puppo, M. Semplice and G. Visconti, CWENO: Uniformly accurate reconstructions for balance laws, Math. Comp., 87 (2018), 1689-1719.  doi: 10.1090/mcom/3273.  Google Scholar

[16]

I. Cravero, M. Semplice and G. Visconti, Optimal definition of the nonlinear weights in multidimensional central WENOZ reconstructions, SIAM J. Numer. Anal., 57 (2019), 2328-2358.  doi: 10.1137/18M1228232.  Google Scholar

[17]

R. Crisovan, D. Torlo, R. Abgrall and S. Tokareva, Model order reduction for parametrized nonlinear hyperbolic problems as an application to uncertainty quantification, J. Comput. Appl. Math., 348 (2019), 466-489.  doi: 10.1016/j.cam.2018.09.018.  Google Scholar

[18]

Y. Deng, Y. Shen, K. Chen and H. Jin, Training recurrent neural network through moment matching for nlp applications, (2018), 3353–3357. Google Scholar

[19]

B. Després, G. Poëtte and D. Lucor, Robust uncertainty propagation in systems of conservation laws with the entropy closure method, Uncertainty Quantification in Computational Fluid Dynamics, Lect. Notes Comput. Sci. Eng., Springer, Heidelberg, 92 (2013), 105–149. doi: 10.1007/978-3-319-00885-1_3.  Google Scholar

[20]

D. Funaro, Polynomial Approximation of Differential Equations, Lecture Notes in Physics. New Series m: Monographs, 8. Springer-Verlag, Berlin, 1992.  Google Scholar

[21]

S. Gerster and M. Herty, Entropies and symmetrization of hyperbolic stochastic Galerkin formulations, Communications in Computational Physics, 27 (2020), 639-671.   Google Scholar

[22]

S. Gerster, M. Herty and A. Sikstel, Hyperbolic stochastic Galerkin formulation for the $p$-system, J. Comput. Phys., 395 (2019), 186-204.  doi: 10.1016/j.jcp.2019.05.049.  Google Scholar

[23]

S. Gerster, M. Herty and H. Yu, Hypocoercivity of stochastic Galerkin formulations for stabilization of kinetic equations, RWTH, preprint, 1–21. Google Scholar

[24]

F. S. Gharehchopogh, Neural networks application in software cost estimation: A case study, in 2011 International Symposium on Innovations in Intelligent Systems and Applications, IEEE, (2011), 69–73. doi: 10.1109/INISTA.2011.5946160.  Google Scholar

[25]

M. B. Giles, Multilevel Monte Carlo methods, Springer Proceedings in Mathematics & Statistics, Monte Carlo and Quasi-Monte Carlo Methods, 65 (2012), 83–103. doi: 10.1007/978-3-642-41095-6_4.  Google Scholar

[26]

P. Glasserman, Monte Carlo Methods in Financial Engineering, Springer, New York, 2004.  Google Scholar

[27]

F. Golse, On the dynamics of large particle systems in the mean field limit, in Macroscopic and Large Scale Phenomena: Coarse Graining, Mean Field Limits and Ergodicity, Lect. Notes Appl. Math. Mech., Springer, 3 (2016), 1–144. doi: 10.1007/978-3-319-26883-5_1.  Google Scholar

[28]

D. Gottlieb and D. Xiu, Galerkin method for wave equations with uncertain coefficients, Commun. Comput. Phys., 3 (2008), 505-518.   Google Scholar

[29]

A. Graves, Practical variational inference for neural networks, NIPS, (2011), 2348–2356. Google Scholar

[30]

E. Haber, F. Lucka and L. Ruthotto, Never look back - A modified EnKF method and its application to the training of neural networks without back propagation, 2018, Preprint, arXiv: 1805.08034. Google Scholar

[31]

M. H. Hassoun et al., Fundamentals of Artificial Neural Networks, MIT press, 1995. Google Scholar

[32]

K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778. doi: 10.1109/CVPR.2016.90.  Google Scholar

[33]

D. O. Hebb, The Organization of Behavior: A Neuropsychological Theory, Psychology Press, 2005. doi: 10.4324/9781410612403.  Google Scholar

[34]

M. Herty, T. Trimborn and G. Visconti, Kinetic theory for residual neural networks, arXiv preprint, arXiv: 2001.04294. Google Scholar

[35]

G. Hu, R. Li and T. Tang, A robust WENO type finite volume solver for steady Euler equations on unstructured grids, Commun. Comput. Phys., 9 (2011), 627-648.  doi: 10.4208/cicp.031109.080410s.  Google Scholar

[36]

J. Hu and S. Jin, A stochastic Galerkin method for the Boltzmann equation with uncertainty, J. Comput. Phys., 315 (2016), 150-168.  doi: 10.1016/j.jcp.2016.03.047.  Google Scholar

[37]

P.-E. Jabin, A review of the mean field limits for vlasov equations, Kinetic & Related Models, 7 (2014), 661-711.  doi: 10.3934/krm.2014.7.661.  Google Scholar

[38]

K. Janocha and W. M. Czarnecki, On loss functions for deep neural networks in classifications, 2017, Preprint, arXiv: 1702.05659v1. Google Scholar

[39]

B. Jiang, T.-Y. H Wu, C. Zheng and W. Wong, Learning summary statistic for approximate bayesian computation via deep neural network, Statistica Sinica, 27 (2017), 1595-1618.   Google Scholar

[40]

G.-S. Jiang and C.-W. Shu, Efficient implementation of weighted ENO schemes, J. Comput. Phys., 126 (1996), 202-228.  doi: 10.1006/jcph.1996.0130.  Google Scholar

[41]

S. Jin and Y. Zhu, Hypocoercivity and uniform regularity for the Vlasov-Poisson-Fokker-Planck system with uncertainty and multiple scales, SIAM J. Math. Anal., 50 (2018), 1790-1816.  doi: 10.1137/17M1123845.  Google Scholar

[42]

M. I. Jordan and T. M. Mitchell, Machine learning: Trends, perspectives, and prospects, Science, 349 (2015), 255-260.  doi: 10.1126/science.aaa8415.  Google Scholar

[43]

A. V. Joshi, Machine learning and artificial intelligence, Springer, Cham, 2020. doi: 10.1007/978-3-030-26622-6.  Google Scholar

[44]

H. M. D. Kabir, A. Khosravi, M. A. Hosen and S. Nahavandi, Neural network-based uncertainty quantification: A survey of methodologies and applications, IEEE Access, 6 (2018), 36218-36234.  doi: 10.1109/ACCESS.2018.2836917.  Google Scholar

[45]

C. Klingenberg, G. Puppo and M. Semplice, Arbitrary order finite volume well-balanced schemes for the Euler equations with gravity, SIAM J. Sci. Comput., 41 (2019), A695–A721. doi: 10.1137/18M1196704.  Google Scholar

[46]

N. B. Kovachki and A. M. Stuart, Ensemble Kalman inversion: A derivative-free technique for machine learning tasks, Inverse Probl., 35 (2019), 095005, 35 pp. doi: 10.1088/1361-6420/ab1c3a.  Google Scholar

[47]

J. Kusch and M. Frank, Intrusive methods in uncertainty quantification and their connection to kinetic theory, Int. J. Adv. Eng. Sci. Appl. Math., 10 (2018), 54-69.  doi: 10.1007/s12572-018-0211-3.  Google Scholar

[48]

O. P. Le Maître and O. M. Knio, Spectral Methods for Uncertainty Quantification, With applications to computational fluid dynamics. Scientific Computation. Springer, New York, 2010. doi: 10.1007/978-90-481-3520-2.  Google Scholar

[49]

C. Li and S. Mahadevan, Efficient approximate inference in Bayesian networks with continuous variables, Reliability Engineering & System Safety, 169 (2018), 269-280.  doi: 10.1016/j.ress.2017.08.017.  Google Scholar

[50]

C.-L. Li, W.-C. Chang, Y. Cheng, Y. Yang and B. Póczos, Mmd gan: Towards deeper understanding of moment matching network, in Advances in Neural Information Processing Systems, (2017), 2203–2213. Google Scholar

[51]

H. Lin and S. Jegelka, Resnet with one-neuron hidden layers is a universal approximator, in Advances in Neural Information Processing Systems (eds. S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett), Curran Associates, Inc., 31 (2018), 6169–6178. Google Scholar

[52]

S. Mandt, M. Hoffman and D. Blei, Stochastic gradient descent as approximate Bayesian inference, J. Mach. Learn. Res., 18 (2017), Paper No. 134, 35 pp.  Google Scholar

[53]

W. S. McCulloch and W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bull. Math. Biophys., 5 (1943), 115-133.  doi: 10.1007/BF02478259.  Google Scholar

[54]

S. Mishra, Ch. Schwab and J. Šukys, Multi-level Monte Carlo finite volume methods for nonlinear systems of conservation laws in multi-dimensions, J. Comput. Phys., 231 (2012), 3365-3388.  doi: 10.1016/j.jcp.2012.01.011.  Google Scholar

[55]

S. Mishra, Ch. Schwab and J. Šukys, Multilevel Monte Carlo finite volume methods for shallow water equations with uncertain topography in multi-dimensions, SIAM J. Sci. Comput., 34 (2012), B761–B784. doi: 10.1137/110857295.  Google Scholar

[56]

O. Nikodym, Sur une généralisation des intégrales de M. J. Radon, Fundamenta Mathematicae, 15 (1930), 131-179.  doi: 10.4064/fm-15-1-131-179.  Google Scholar

[57]

S. C. Onar, A. Ustundag, Ç. Kadaifci and B. Oztaysi, The changing role of engineering education in industry 4.0 era, in Industry 4.0: Managing The Digital Transformation, Springer, (2018), 137–151. Google Scholar

[58]

P. Pettersson, G. Iaccarino and J. Nordström, A stochastic Galerkin method for the Euler equations with Roe variable transformation, Journal of Computational Physics, 257 (2014), 481-500.  doi: 10.1016/j.jcp.2013.10.011.  Google Scholar

[59]

G. Poëtte, B. Després and D. Lucor, Uncertainty quantification for systems of conservation laws, J. Comput. Phys., 228 (2009), 2443-2467.  doi: 10.1016/j.jcp.2008.12.018.  Google Scholar

[60]

R. Pulch and E. Maten, Stochastic galerkin methods and model order reduction for linear dynamical systems, International Journal for Uncertainty Quantification, 5 (2015), 255-273.  doi: 10.1615/Int.J.UncertaintyQuantification.2015010171.  Google Scholar

[61]

R. Pulch and D. Xiu, Generalised polynomial chaos for a class of linear conservation laws, J. Sci. Comput., 51 (2012), 293-312.  doi: 10.1007/s10915-011-9511-5.  Google Scholar

[62]

D. Ray and J. S. Hesthaven, An artificial neural network as a troubled-cell indicator, J. Comput. Phys., 367 (2018), 166-191.  doi: 10.1016/j.jcp.2018.04.029.  Google Scholar

[63]

D. Ray and J. S. Hesthaven, Detecting troubled-cells on two-dimensional unstructured grids using a neural network, J. Comput. Phys., 397 (2019), 108845, 31 pp. doi: 10.1016/j.jcp.2019.07.043.  Google Scholar

[64]

N. H. Risebro, C. Schwab and F. Weber, Multilevel Monte Carlo front-tracking for random scalar conservation laws, BIT, 56 (2016), 263-292.  doi: 10.1007/s10543-015-0550-4.  Google Scholar

[65]

L. Ruthotto, S. Osher, W. Li, L. Nurbekyan and S. W. Fung, A machine learning framework for solving high-dimensional mean field game and mean field control problems, Proceedings of the National Academy of Sciences, 117, (2020), 9183–9193. Google Scholar

[66]

L. Ruthotto and E. Haber, Deep neural networks motivated by partial differential equations, J. Math. Imaging Vision, 62 (2020), 352-364.  doi: 10.1007/s10851-019-00903-1.  Google Scholar

[67]

G. Scarciotti and A. Astolfi, Data-driven model reduction by moment matching for linear and nonlinear systems, Automatica J. IFAC, 79 (2017), 340-351.  doi: 10.1016/j.automatica.2017.01.014.  Google Scholar

[68]

R. Schmitt and G. Schuh, Advances in Production Research: Proceedings of the 8th Congress of the German Academic Association for Production Technology (WGP), Aachen, November 19-20, 2018, Springer, 2018. doi: 10.1007/978-3-030-03451-1.  Google Scholar

[69]

C. Schwab and S. Tokareva, High order approximation of probabilistic shock profiles in hyperbolic conservation laws with uncertain initial data, ESAIM Math. Model. Numer. Anal., 47 (2013), 807-835.  doi: 10.1051/m2an/2012060.  Google Scholar

[70]

M. Semplice, A. Coco and G. Russo, Adaptive mesh refinement for hyperbolic systems based on third-order compact WENO reconstruction, J. Sci. Comput., 66 (2016), 692-724.  doi: 10.1007/s10915-015-0038-z.  Google Scholar

[71]

R. Shu, J. Hu and S. Jin, A stochastic Galerkin method for the Boltzmann equation with multi-dimensional random inputs using sparse wavelet bases, Numer. Math. Theory Methods Appl., 10 (2017), 465-488.  doi: 10.4208/nmtma.2017.s12.  Google Scholar

[72]

R. C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2013. Google Scholar

[73]

R. C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, Computational Science & Engineering, 12. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2014.  Google Scholar

[74]

R. J. Solomonoff, Machine Learning – Past and Future, The Dartmouth Artificial Intelligence Conference, 2006. Google Scholar

[75]

J. Stoer and R. Bulirsch, Introduction to Numerical Analysis, 3rd edition, Texts in Applied Mathematics, 12. Springer-Verlag New York, 2002. doi: 10.1007/978-0-387-21738-3.  Google Scholar

[76]

J. Šukys, S. Mishra and C. Schwab, Multi-level Monte Carlo finite difference and finite volume methods for stochastic linear hyperbolic systems, Springer Proceedings in Mathematics and Statistics, 65 (2013), 649-666.  doi: 10.1007/978-3-642-41095-6_34.  Google Scholar

[77]

T. Tang and T. Zhou, Convergence analysis for stochastic collocation methods to scalar hyperbolic equations with a random wave speed, Commun. Comput. Phys., 8 (2010), 226-248.  doi: 10.4208/cicp.060109.130110a.  Google Scholar

[78]

H. Tercan, T. Al Khawli, U. Eppelt, C. Büscher, T. Meisen and S. Jeschke, Improving the laser cutting process design by machine learning techniques, Production Engineering, 11 (2017), 195-203.  doi: 10.1007/s11740-017-0718-7.  Google Scholar

[79]

H. H. Thodberg, A review of bayesian neural networks with an application to near infrared spectroscopy, IEEE Transactions on Neural Networks, 7 (1996), 56-72.   Google Scholar

[80]

S. Tokareva, C. Schwab and S. Mishra, High order SFV and mixed SDG/FV methods for the uncertainty quantification in multidimensional conservation laws, in High Order Nonlinear Numerical Schemes for Evolutionary PDEs, Lect. Notes Comput. Sci. Eng., Springer, Cham, 99 (2014), 109–133.  Google Scholar

[81]

R. K. Tripathy and I. Bilionis, Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification, J. Comput. Phys., 375 (2018), 565-588.  doi: 10.1016/j.jcp.2018.08.036.  Google Scholar

[82]

V. Vanhoucke, A. Senior and M. Z. Mao, Improving the Speed of Neural Networks on CPUs., Deep Learning and Unsupervised Feature Learning Workshop, 2011. Google Scholar

[83]

Q. Wang, J. S. Hesthaven and D. Ray, Non-intrusive reduced order modelling of unsteady flows using artificial neural networks with application to a combustion problem, J. Comput. Phys., 384 (2019), 289-307.  doi: 10.1016/j.jcp.2019.01.031.  Google Scholar

[84]

K. Watanabe and S. G. Tzafestas, Learning algorithms for neural networks with the Kalman filters, J. Intell. Robot. Syst., 3 (1990), 305-319.  doi: 10.1007/BF00439421.  Google Scholar

[85]

N. Wiener, The homogeneous chaos, Amer. J. Math., 60 (1938), 897-936.  doi: 10.2307/2371268.  Google Scholar

[86]

D. Xiu, Numerical methods for stochastic computations, A spectral method approach. Princeton University Press, Princeton, 2010.  Google Scholar

[87]

D. Xiu and G. E. Karniadakis, The Wiener-Askey polynomial chaos for stochastic differential equations, SIAM J. Sci. Comput., 24 (2002), 619-644.  doi: 10.1137/S1064827501387826.  Google Scholar

[88]

M. Zanella, Structure preserving stochastic Galerkin methods for Fokker-Planck equations with background interactions, Math. Comput. Simulation, 168 (2020), 28-47.  doi: 10.1016/j.matcom.2019.07.012.  Google Scholar

[89]

Y. Zhu and S. Jin, The Vlasov-Poisson-Fokker-Planck system with uncertainty and a one-dimensional asymptotic preserving method, Multiscale Model. Simul., 15 (2017), 1502-1529.  doi: 10.1137/16M1090028.  Google Scholar

Figure 1.  Illustration of a neural network with $ L+1 = 5 $ total layers
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Illustration of the SimResNet with $ L+1 = 5 $ total layers
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  Top left panel: Regression of the target function $ h(x) = \tanh(x) $ on $ \mathbb{H} = [-4,4] $ provided by the deep artificial neural networks with the statistical quantities computed with the Monte-Carlo method and the stochastic collocation method with noise level $ \varepsilon = 0.3 $. Top right panel: Expected value of the quantity of interest. Bottom left panel: Variance of the quantity of interest. Bottom right panel: Convergence rate of the Monte-Carlo method
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  Top left panel: classification problem with the statistical quantities computed with the Monte-Carlo method and the stochastic collocation method with noise level $ \varepsilon = 0.5 $. Top right panel: expected value of the quantity of interest. Bottom left panel: variance of the quantity of interest. Bottom right panel: convergence rate of the Monte-Carlo method
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  CASE 1: Projection of initial data and root mean squared error (RMSE) with exactly computed mean and variance. Upper panel: The left $ y $-axis shows the gPC modes $ \hat{g}_k(0,x) := \big\langle g_0(x,\cdot ) \, \phi_k( \cdot ) \big\rangle_\rho $ of initial values. The mean $ \mathbb{E}\big[ g_0({x},\boldsymbol{\eta}) \big] = \hat{g}_0(0,x) $ is plotted in red, the others in green. The confidence region, which contains $ \mathbb{P} $-a.s. all realisations $ g_0\big(x,\boldsymbol{\eta}(\omega)\big) $ is gray shaded. The right $ y $-axis shows the variance corresponding to different gPC truncations. The upper left panel shows for each fixed signal $ x $ the root mean squared error (RMSE) of the gPC truncations. The right panel shows the total RMSE for various gPC truncations
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  CASE 2: Projection of initial data and root mean squared error (RMSE) with exactly computed mean and variance. Upper panel: The blue lines describe deviations from the mean of initial data (red). Two realisations are shown in black, which are in the confidence region (gray shaded). The lower left panel shows for each fixed signal $ x $ the root mean squared error (RMSE) of the gPC truncations. The right panel shows the total RMSE for various gPC truncations
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7.  CASE 1: Solutions to the stochastic Galerkin formulation with truncation $ K = 30 $ and discretization $ \Delta x = 0.01 $. The upper panels show in red the first gPC mode $ \hat{g}_0(t,x) $, which describes the temporal evolution of the mean. The other modes are plotted in green, which determine the spread of the stochastic perturbations. The lower panels show the probability distributions of the random quantity $ g(t,x,\boldsymbol{\eta}) $ for a fixed signal $ x\in\{2,8\} $ at time $ t = 0 $ (left), $ t = 0.5 $ (middle) and $ t = 1 $ (right)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 8.  CASE 2: Solutions to the stochastic Galerkin formulation with truncation $ K = 10 $ and discretization $ \Delta x = 0.01 $. The left $ y $-axes of the first and second panel show the mean and deviations. The right $ y $-axes show the gPC modes $ \hat{g}_1,\ldots,\hat{g}_3 $. The right panels show the probability distributions of the random quantity $ g(t,1,\boldsymbol{\eta}) $ at time $ t = 0 $ (left), $ t = 0 $ and $ t = 5 $
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 9.  CASE 1: Mean, variance and distribution for random loss. The left panels show for each fixed signal $ x $ the mean and the variance of the random loss. The middle panels show for various noise levels $ \varepsilon $ the mean and variance of the total random loss. The right panel shows the probability distributions of the random loss in $ x = 2 $ and $ x = 8 $ at time $ t = 0.5 $ and $ t = 1 $
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 10.  CASE 2: Mean, variance and distribution for random loss. The left panels show for each fixed signal $ x $ the mean and the variance of the random loss. The middle panels show for various noise levels $ \varepsilon $ the mean and variance of the total random loss. The right panels show the probability distributions of the random loss in $ x = 1 $, respectively at time $ t = 0 $ and $ t = 5 $
Figure Options

Download full-size image

Download as PowerPoint slide

[1]

Benoît Perthame, Delphine Salort. On a voltage-conductance kinetic system for integrate & fire neural networks. Kinetic & Related Models, 2013, 6 (4) : 841-864. doi: 10.3934/krm.2013.6.841
[2]

Andrew J. Majda, Michal Branicki. Lessons in uncertainty quantification for turbulent dynamical systems. Discrete & Continuous Dynamical Systems - A, 2012, 32 (9) : 3133-3221. doi: 10.3934/dcds.2012.32.3133
[3]

Jing Li, Panos Stinis. Mori-Zwanzig reduced models for uncertainty quantification. Journal of Computational Dynamics, 2019, 6 (1) : 39-68. doi: 10.3934/jcd.2019002
[4]

H. T. Banks, Robert Baraldi, Karissa Cross, Kevin Flores, Christina McChesney, Laura Poag, Emma Thorpe. Uncertainty quantification in modeling HIV viral mechanics. Mathematical Biosciences & Engineering, 2015, 12 (5) : 937-964. doi: 10.3934/mbe.2015.12.937
[5]

Alex Capaldi, Samuel Behrend, Benjamin Berman, Jason Smith, Justin Wright, Alun L. Lloyd. Parameter estimation and uncertainty quantification for an epidemic model. Mathematical Biosciences & Engineering, 2012, 9 (3) : 553-576. doi: 10.3934/mbe.2012.9.553
[6]

Ryan Bennink, Ajay Jasra, Kody J. H. Law, Pavel Lougovski. Estimation and uncertainty quantification for the output from quantum simulators. Foundations of Data Science, 2019, 1 (2) : 157-176. doi: 10.3934/fods.2019007
[7]

Sylvia Novo, Rafael Obaya, Ana M. Sanz. Exponential stability in non-autonomous delayed equations with applications to neural networks. Discrete & Continuous Dynamical Systems - A, 2007, 18 (2&3) : 517-536. doi: 10.3934/dcds.2007.18.517
[8]

Yuhua Zhu. A local sensitivity and regularity analysis for the Vlasov-Poisson-Fokker-Planck system with multi-dimensional uncertainty and the spectral convergence of the stochastic Galerkin method. Networks & Heterogeneous Media, 2019, 14 (4) : 677-707. doi: 10.3934/nhm.2019027
[9]

Ying Sue Huang. Resynchronization of delayed neural networks. Discrete & Continuous Dynamical Systems - A, 2001, 7 (2) : 397-401. doi: 10.3934/dcds.2001.7.397
[10]

Yanqing Wang. A semidiscrete Galerkin scheme for backward stochastic parabolic differential equations. Mathematical Control & Related Fields, 2016, 6 (3) : 489-515. doi: 10.3934/mcrf.2016013
[11]

Émilie Chouzenoux, Henri Gérard, Jean-Christophe Pesquet. General risk measures for robust machine learning. Foundations of Data Science, 2019, 1 (3) : 249-269. doi: 10.3934/fods.2019011
[12]

Ruoxia Li, Huaiqin Wu, Xiaowei Zhang, Rong Yao. Adaptive projective synchronization of memristive neural networks with time-varying delays and stochastic perturbation. Mathematical Control & Related Fields, 2015, 5 (4) : 827-844. doi: 10.3934/mcrf.2015.5.827
[13]

Pierre Guiraud, Etienne Tanré. Stability of synchronization under stochastic perturbations in leaky integrate and fire neural networks of finite size. Discrete & Continuous Dynamical Systems - B, 2019, 24 (9) : 5183-5201. doi: 10.3934/dcdsb.2019056
[14]

Sun Yi, Patrick W. Nelson, A. Galip Ulsoy. Delay differential equations via the matrix lambert w function and bifurcation analysis: application to machine tool chatter. Mathematical Biosciences & Engineering, 2007, 4 (2) : 355-368. doi: 10.3934/mbe.2007.4.355
[15]

Tatyana S. Turova. Structural phase transitions in neural networks. Mathematical Biosciences & Engineering, 2014, 11 (1) : 139-148. doi: 10.3934/mbe.2014.11.139
[16]

José Miguel Pasini, Tuhin Sahai. Polynomial chaos based uncertainty quantification in Hamiltonian, multi-time scale, and chaotic systems. Journal of Computational Dynamics, 2014, 1 (2) : 357-375. doi: 10.3934/jcd.2014.1.357
[17]

Jiang Xie, Junfu Xu, Celine Nie, Qing Nie. Machine learning of swimming data via wisdom of crowd and regression analysis. Mathematical Biosciences & Engineering, 2017, 14 (2) : 511-527. doi: 10.3934/mbe.2017031
[18]

Mingbao Cheng, Shuxian Xiao, Guosheng Liu. Single-machine rescheduling problems with learning effect under disruptions. Journal of Industrial & Management Optimization, 2018, 14 (3) : 967-980. doi: 10.3934/jimo.2017085
[19]

Andreas Chirstmann, Qiang Wu, Ding-Xuan Zhou. Preface to the special issue on analysis in machine learning and data science. Communications on Pure & Applied Analysis, 2020, 19 (8) : i-iii. doi: 10.3934/cpaa.2020171
[20]

Qi Lü, Xu Zhang. Transposition method for backward stochastic evolution equations revisited, and its application. Mathematical Control & Related Fields, 2015, 5 (3) : 529-555. doi: 10.3934/mcrf.2015.5.529

 Impact Factor: 

Tools

Article outline

Figures and Tables

[Back to Top]

Copyright © 2020 American Institute of Mathematical Sciences