An operator learning perspective on parameter-to-observable maps

Daniel Zhengyu Huang; Nicholas H. Nelsen; Margaret Trautner

doi:10.3934/fods.2024037

Article Contents

2024. Doi: 10.3934/fods.2024037

Previous Article Next Article

Early Access

Early Access articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Early Access publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Early Access articles via the “Early Access” tab for the selected journal.

An operator learning perspective on parameter-to-observable maps

1.
Beijing International Center for Mathematical Research, Peking University, Beijing, China
2.
Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA, USA

^*Corresponding author: Nicholas H. Nelsen (nnelsen@caltech.edu)
^*Corresponding author: Nicholas H. Nelsen (nnelsen@caltech.edu)

Received: January 2024

Revised: June 2024

Early access: August 2024

Abstract / Introduction Full Text(HTML) Figure(10) / Table(1) Related Papers Cited by

Abstract

Computationally efficient surrogates for parametrized physical models play a crucial role in science and engineering. Operator learning provides data-driven surrogates that map between function spaces. However, instead of full-field measurements, often the available data are only finite-dimensional parametrizations of model inputs or finite observables of model outputs. Building on Fourier Neural Operators, this paper introduces the Fourier Neural Mappings (FNMs) framework that is able to accommodate such finite-dimensional vector inputs or outputs. The paper develops universal approximation theorems for the method. Moreover, in many applications the underlying parameter-to-observable (PtO) map is defined implicitly through an infinite-dimensional operator, such as the solution operator of a partial differential equation. A natural question is whether it is more data-efficient to learn the PtO map end-to-end or first learn the solution operator and subsequently compute the observable from the full-field solution. A theoretical analysis of Bayesian nonparametric regression of linear functionals, which is of independent interest, suggests that the end-to-end approach can actually have worse sample complexity. Extending beyond the theory, numerical results for the FNM approximation of three nonlinear PtO maps demonstrate the benefits of the operator learning perspective that this paper adopts.

Keywords:

Mathematics Subject Classification: Primary: 68T07, 62G20; Secondary: 65J15.

Citation:

FullText(HTML)

Figure 1. Illustration of the factorization of an underlying PtO map into a QoI and an operator between function spaces. Also shown are the four variants of input and output representations considered in this work. Here, $ \mathcal{U} $ is an input function space and $ \mathcal{Y} $ is an intermediate function space

Download: Full-size image PowerPoint slide

Figure 2. (EE) vs. (FF) convergence rate exponents (40) as a function of QoI regularity exponent $ r $. Larger exponents imply faster convergence rates. As the curves gets lighter, $ \alpha+\beta $, an indicator of the smoothness of the problem, increases. The vertical dashed line corresponds to $ r = -1/2 $, which is the transition point where (EE) and (FF) have the same rate and the onset of power law decay for the QoI coefficients begins

Download: Full-size image PowerPoint slide

Figure 3. Empirical sample complexity of the Bayesian (EE) and (FF) estimators for linear PtO maps based on a Poisson problem. The solid purple lines are best linear fits to the broken curves with markers, which correspond to numerically computed squared errors. In all three figures, the experimentally observed convergence rates are nearly perfect matches to those from the theoretical upper bounds in Corollary 4.11 (see Table 1)

Download: Full-size image PowerPoint slide

Figure 4. Visualization of the velocity-to-state map for the advection–diffusion model. Rows denote the dimension of the KL expansion of the velocity profile and columns display representative input and output fields

Download: Full-size image PowerPoint slide

Figure 5. Empirical sample complexity of FNM and NN architectures for the advection–diffusion PtO map (note that Figure 5a has a different vertical axis range). The shaded regions denote two standard deviations away from the mean of the test error over $ 5 $ realizations of the random training dataset indices, batch indices during SGD, and model parameter initializations

Download: Full-size image PowerPoint slide

Figure 6. Flow over an airfoil. From left to right: visualization of the cubic design element and different airfoil configurations, guided by the displacement field of the control nodes; a close-up view of the $ C $-grid surrounding the airfoil; the physical domain discretized by the $ C $-grid

Download: Full-size image PowerPoint slide

Figure 7. Flow over an airfoil. The 1D (bottom) and 2D (top) latent spaces are illustrated at the center; the input functions $ \phi_a $ encoding the irregular physical domains, are shown on the left; and the output functions $ p\circ\phi_a $ representing the pressure field on the irregular physical domains, are depicted on the right

Download: Full-size image PowerPoint slide

Figure 8. Flow over an airfoil. Comparative analysis of relative test error versus data size for the FNM and NN approaches. The shaded regions denote two standard deviations away from the mean of the test error over $ 5 $ realizations of the batch indices during SGD and model parameter initializations

Download: Full-size image PowerPoint slide

Figure 9. Diagram showing the homogenization experiment ground truth maps. The function $ A $ is parametrized by a finite vector $ z $. The quantity of interest $ \overline{A} $ (56) is computed from both the material function $ A $ and the solution $ \chi $ to the cell problem (57). Note that both $ A $ and $ \chi $ are functions on the torus $ \mathbb{T}^2 $

Download: Full-size image PowerPoint slide

Figure 10. Elliptic homogenization problem. Absolute $ \overline{A} $ error in the Frobenius norm versus data size for the FNM and NN architectures. The shaded regions denote two standard deviations away from the mean of the test error over $ 5 $ realizations of batch indices during SGD and model parameter initializations

Download: Full-size image PowerPoint slide

Table 1. Sample complexity comparison experiment. The entries of the table record the theoretical vs. experimental (in parentheses) convergence rate exponents $ \rho $ in $ O(N^{-\rho}) $ of the relative expected squared $ L_{\nu}^{2}(H; \mathbb{R}) $ in-distribution error (44). The theoretical rates are taken from the upper bound in Corollary 4.11. Here $ \alpha = 0.75 $

	$ r=-3/2 $	$ r=-1/2 $	$ r=1/2 $
$ \mathrm{EE} $	0.714 (0.738)	0.818 (0.815)	0.867 (0.864)
$ \mathrm{FF} $	0.455 (0.455)	0.818 (0.818)	1.000 (1.003)

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	B. Adcock, N. Dexter and S. Moraga, Optimal approximation of infinite-dimensional holomorphic functions, Calcolo, 61 (2024), Paper No. 12, 45 pp. doi: 10.1007/s10092-023-00565-x.
[2]	S. Agapiou and I. Castillo, Heavy-tailed Bayesian nonparametric adaptation, preprint, arXiv: 2308.04916.
[3]	S. Agapiou, S. Larsson and A. M. Stuart, Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems, Stochastic Processes and their Applications, 123 (2013), 3828-3860. doi: 10.1016/j.spa.2013.05.001.
[4]	S. Agapiou, A. M. Stuart and Y.-X. Zhang, Bayesian posterior contraction rates for linear severely ill-posed inverse problems, Journal of Inverse and Ill-Posed Problems, 22 (2014), 297-321. doi: 10.1515/jip-2012-0071.
[5]	P. L. Bartlett, P. M. Long, G. Lugosi and A. Tsigler, Benign overfitting in linear regression, Proceedings of the National Academy of Sciences, 117 (2020), 30063-30070. doi: 10.1073/pnas.1907378117.
[6]	J.-H. Bastek and D. M. Kochmann, Inverse-design of nonlinear mechanical metamaterials via video denoising diffusion models, Nature Machine Intelligence, 5 (2023), 1466-1475. doi: 10.1038/s42256-023-00762-x.
[7]	A. Bensoussan, J.-L. Lions and G. Papanicolaou, Asymptotic Analysis for Periodic Structures, vol. 374, American Mathematical Society, 2011. doi: 10.1090/chel/374.
[8]	K. Bhattacharya, N. B. Kovachki, A. Rajan, A. M. Stuart and M. Trautner, Learning homogenization for elliptic operators, SIAM Journal on Numerical Analysis, 62 (2024), 1844-1873. doi: 10.1137/23M1585015.
[9]	N. Boullé, D. Halikias and A. Townsend, Elliptic PDE learning is provably data-efficient, Proceedings of the National Academy of Sciences, 120 (2023), Paper No. e2303904120, 3 pp. doi: 10.1073/pnas.2303904120.
[10]	T. T. Cai and P. Hall, Prediction in functional linear regression, Annals of Statistics, 34 (2006), 2159-2179. doi: 10.1214/009053606000000830.
[11]	T. T. Cai and M. Yuan, Minimax and adaptive prediction for functional linear regression, Journal of the American Statistical Association, 107 (2012), 1201-1216. doi: 10.1080/01621459.2012.716337.
[12]	A. Caponnetto and E. De Vito, Optimal rates for the regularized least-squares algorithm, Foundations of Computational Mathematics, 7 (2007), 331-368. doi: 10.1007/s10208-006-0196-8.
[13]	H. Cardot, A. Mas and P. Sarda, CLT in functional linear regression models, Probability Theory and Related Fields, 138 (2007), 325-361. doi: 10.1007/s00440-006-0025-2.
[14]	T. Chen and H. Chen, Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems, IEEE Transactions on Neural Networks, 6 (1995), 911-917. doi: 10.1109/72.392253.
[15]	G. Cybenko, Approximation by superpositions of a sigmoidal function, Mathematics of Control, Signals and Systems, 2 (1989), 303-314. doi: 10.1007/BF02551274.
[16]	M. Dashti and A. M. Stuart, The Bayesian approach to inverse problems, in Handbook of Uncertainty Quantification (eds. R. Ghanem, D. Higdon and H. Owhadi), Springer, Cham, 2017, 311-428. doi: 10.1007/978-3-319-12385-1_7.
[17]	M. V. de Hoop, N. B. Kovachki, N. H. Nelsen and A. M. Stuart, Convergence rates for learning linear operators from noisy data, SIAM/ASA Journal on Uncertainty Quantification, 11 (2023), 480-513.
[18]	B. Deng, Y. Shin, L. Lu, Z. Zhang and G. E. Karniadakis, Approximation rates of DeepONets for learning operators arising from advection–diffusion equations, Neural Networks, 153 (2022), 411-426. doi: 10.1016/j.neunet.2022.06.019.
[19]	J. Dugundji, An extension of Tietze's theorem, Pacific Journal of Mathematics, 1 (1951), 353-367. doi: 10.2140/pjm.1951.1.353.
[20]	W. E, C. Ma and L. Wu, The Barron space and the flow-induced function spaces for neural network models, Constructive Approximation, 55 (2022), 369-406. doi: 10.1007/s00365-021-09549-y.
[21]	G. Farin, Curves and Surfaces for Computer-Aided Geometric Design: A Practical Guide, Elsevier, 2014.
[22]	S. Fischer and I. Steinwart, Sobolev norm learning rates for regularized least-squares algorithms, The Journal of Machine Learning Research, 21 (2020), Paper No. 205, 38 pp.
[23]	M. B. Giles and E. Süli, Adjoint methods for PDEs: A posteriori error analysis and postprocessing by duality, Acta Numerica, 11 (2002), 145-236. doi: 10.1017/S096249290200003X.
[24]	S. Goswami, M. Yin, Y. Yu and G. E. Karniadakis, A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials, Computer Methods in Applied Mechanics and Engineering, 391 (2022), Paper No. 114587, 29 pp. doi: 10.1016/j.cma.2022.114587.
[25]	S. Gugushvili, A. van der Vaart and D. Yan, Bayesian linear inverse problems in regularity scales, Annales de l'Institut Henri Poincaré-Probabilités et Statistiques, 56 (2020), 2081-2107. doi: 10.1214/19-AIHP1029.
[26]	E. Hasani and R. A. Ward, Generating synthetic data for neural operators, preprint, arXiv: 2401.02398.
[27]	L. Herrmann, J. A. A. Opschoor and C. Schwab, Constructive deep ReLU neural network approximation, Journal of Scientific Computing, 90 (2022), Paper No. 75, 37 pp. doi: 10.1007/s10915-021-01718-2.
[28]	L. Herrmann, C. Schwab and J. Zech, Neural and GPC operator surrogates: Construction and expression rate bounds, preprint, arXiv: 2207.04950.
[29]	K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks, 4 (1991), 251-257. doi: 10.1016/0893-6080(91)90009-T.
[30]	L. Hucker and M. Wahl, A note on the prediction error of principal component regression in high dimensions, Theory of Probability and Mathematical Statistics, 109 (2023), 37-53. doi: 10.1090/tpms/1196.
[31]	J. Jin, Y. Lu, J. Blanchet and L. Ying, Minimax optimal kernel operator learning via multilevel training, in The Eleventh International Conference on Learning Representations, 2022.
[32]	B. Knapik and J.-B. Salomond, A general approach to posterior contraction in nonparametric inverse problems, Bernoulli, 24 (2018), 2091-2121. doi: 10.3150/16-BEJ921.
[33]	B. T. Knapik, B. T. Szabó, A. W. Van Der Vaart and J. H. van Zanten, Bayes procedures for adaptive inference in inverse problems for the white noise model, Probability Theory and Related Fields, 164 (2016), 771-813. doi: 10.1007/s00440-015-0619-7.
[34]	B. T. Knapik, A. W. Van Der Vaart and J. H. van Zanten, Bayesian inverse problems with Gaussian priors, Annals of Statistics, 39 (2011), 2626-2657. doi: 10.1214/11-AOS920.
[35]	N. Kovachki, S. Lanthaler and S. Mishra, On universal approximation and error bounds for Fourier neural operators, The Journal of Machine Learning Research, 22 (2021), Paper No. [290], 76 pp.
[36]	N. B. Kovachki, S. Lanthaler and H. Mhaskar, Data complexity estimates for operator learning, preprint, arXiv: 2405.15992.
[37]	N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart and A. Anandkumar, Neural operator: Learning maps between function spaces with applications to PDEs, Journal of Machine Learning Research, 24 (2023), Paper No. [89], 97 pp.
[38]	T. Kurth, S. Subramanian, P. Harrington, J. Pathak, M. Mardani, D. Hall, A. Miele, K. Kashinath and A. Anandkumar, FourCastNet: Accelerating global high-resolution weather forecasting using adaptive Fourier neural operators, in Proceedings of the Platform for Advanced Scientific Computing Conference, (2023), Article No.: 13, 1-11. doi: 10.1145/3592979.3593412.
[39]	S. Lanthaler, Operator learning with PCA-Net: Upper and lower complexity bounds, Journal of Machine Learning Research, 24 (2023), Paper No. [318], 67 pp.
[40]	S. Lanthaler, Z. Li and A. M. Stuart, The nonlocal neural operator: Universal approximation, preprint, arXiv: 2304.13221.
[41]	S. Lanthaler, R. Molinaro, P. Hadorn and S. Mishra, Nonlinear reconstruction for operator learning of PDEs with discontinuities, in The Eleventh International Conference on Learning Representations, 2022.
[42]	S. Lanthaler and N. H. Nelsen, Error bounds for learning with vector-valued random features, in Advances in Neural Information Processing Systems (eds. A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt and S. Levine), Curran Associates, Inc., 36 (2023), 71834-71861.
[43]	S. Lanthaler and A. M. Stuart, The parametric complexity of operator learning, preprint, arXiv: 2306.15924.
[44]	S. Lanthaler, A. M. Stuart and M. Trautner, Discretization error of Fourier neural operators, preprint, arXiv: 2405.02221.
[45]	T. Laurent and J. Brecht, Deep linear networks with arbitrary loss: All local minima are global, in International Conference on Machine Learning, PMLR, 2018, 2902-2907.
[46]	Z. Li, D. Z. Huang, B. Liu and A. Anandkumar, Fourier neural operator with learned deformations for PDEs on general geometries, Journal of Machine Learning Research, 24 (2023), Paper No. [388], 26 pp.
[47]	Z. Li, N. B. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. M. Stuart and A. Anandkumar, Neural operator: Graph kernel network for partial differential equations, preprint, arXiv: 2003.03485.
[48]	Z. Li, N. B. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. M. Stuart and A. Anandkumar, Fourier neural operator for parametric partial differential equations, International Conference on Learning Representations.
[49]	Z. Li, N. B. Kovachki, K. Azizzadenesheli, B. Liu, A. M. Stuart, K. Bhattacharya and A. Anandkumar, Multipole graph neural operator for parametric partial differential equations, in Advances in Neural Information Processing Systems (eds. H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan and H. Lin), Curran Associates, Inc., 33 (2020), 6755-6766.
[50]	Z. Li, D. Meunier, M. Mollenhauer and A. Gretton, Towards optimal Sobolev norm rates for the vector-valued regularized least-squares algorithm, preprint, arXiv: 2312.07186.
[51]	H. Lian, T. Choi, J. Meng and S. Jo, Posterior convergence for Bayesian functional linear regression, Journal of Multivariate Analysis, 150 (2016), 27-41. doi: 10.1016/j.jmva.2016.04.008.
[52]	L. Lingsch, M. Michelis, S. M. Perera, R. K. Katzschmann and S. Mishra, A structured matrix method for nonequispaced neural operators, preprint, arXiv: 2305.19663.
[53]	B. Liu, N. Kovachki, Z. Li, K. Azizzadenesheli, A. Anandkumar, A. M. Stuart and K. Bhattacharya, A learning-based multiscale method and its application to inelastic impact problems, Journal of the Mechanics and Physics of Solids, 158 (2022), Paper No. 104668, 16 pp. doi: 10.1016/j.jmps.2021.104668.
[54]	H. Liu, H. Yang, M. Chen, T. Zhao and W. Liao, Deep nonparametric estimation of operators between infinite dimensional spaces, Journal of Machine Learning Research, 25 (2024), Paper No. [24], 67 pp.
[55]	J. Lu, Z. Shen, H. Yang and S. Zhang, Deep network approximation for smooth functions, SIAM Journal on Mathematical Analysis, 53 (2021), 5465-5506. doi: 10.1137/20M134695X.
[56]	L. Lu, P. Jin, G. Pang, Z. Zhang and G. E. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence, 3 (2021), 218-229. doi: 10.1038/s42256-021-00302-5.
[57]	K. O. Lye, S. Mishra and R. Molinaro, A multi-level procedure for enhancing accuracy of machine learning algorithms, European Journal of Applied Mathematics, 32 (2021), 436-469. doi: 10.1017/S0956792520000224.
[58]	K. O. Lye, S. Mishra and D. Ray, Deep learning observables in computational fluid dynamics, Journal of Computational Physics, 410 (2020), 109339, 26 pp. doi: 10.1016/j.jcp.2020.109339.
[59]	K. O. Lye, S. Mishra, D. Ray and P. Chandrashekar, Iterative surrogate model optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks, Computer Methods in Applied Mechanics and Engineering, 374 (2021), Paper No. 113575, 27 pp. doi: 10.1016/j.cma.2020.113575.
[60]	S. Mishra and T. K. Rusch, Enhancing accuracy of deep learning algorithms by training with low-discrepancy sequences, SIAM Journal on Numerical Analysis, 59 (2021), 1811-1834. doi: 10.1137/20M1344883.
[61]	M. Mollenhauer, N. Mücke and T. J. Sullivan, Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem, preprint, arXiv: 2211.08875.
[62]	N. H. Nelsen, Statistical Foundations of Operator Learning, PhD thesis, California Institute of Technology, 2024.
[63]	N. H. Nelsen and A. M. Stuart, The random feature model for input-output maps between Banach spaces, SIAM Journal on Scientific Computing, 43 (2021), A3212-A3243. doi: 10.1137/20M133957X.
[64]	N. H. Nelsen and A. M. Stuart, Operator learning using random features: A tool for scientific computing, SIAM Review, 66 (2024), 535-571. doi: 10.1137/24M1648703.
[65]	H. Ogawa, An operator pseudo-inversion lemma, SIAM Journal on Applied Mathematics, 48 (1988), 1527-1531. doi: 10.1137/0148095.
[66]	T. O'Leary-Roseberry, X. Du, A. Chaudhuri, J. R. R. A. Martins, K. Willcox and O. Ghattas, Learning high-dimensional parametric maps via reduced basis adaptive residual networks, Computer Methods in Applied Mechanics and Engineering, 402 (2022), Paper No. 115730, 29 pp. doi: 10.1016/j.cma.2022.115730.
[67]	R. G. Patel, N. A. Trask, M. A. Wood and E. C. Cyr, A physics-informed operator regression framework for extracting data-driven continuum models, Computer Methods in Applied Mechanics and Engineering, 373 (2021), Paper No. 113500, 23 pp. doi: 10.1016/j.cma.2020.113500.
[68]	D. Pati, A. Bhattacharya and G. Cheng, Optimal Bayesian estimation in random covariate design with a rescaled Gaussian process prior, Journal of Machine Learning Research, 16 (2015), 2837-2851.
[69]	G. Pavliotis and A. M. Stuart, Multiscale Methods: Averaging and Homogenization, Springer Science & Business Media, 2008.
[70]	I. F. Pinelis and A. I. Sakhanenko, Remarks on inequalities for large deviation probabilities, Theory of Probability & its Applications, 30 (1986), 143-148. doi: 10.1137/1130013.
[71]	M. A. Rahman, M. A. Florez, A. Anandkumar, Z. E. Ross and K. Azizzadenesheli, Generative adversarial neural operators, Transactions on Machine Learning Research.
[72]	C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning, vol. 1, Springer, 2006. doi: 10.7551/mitpress/3206.001.0001.
[73]	K. Ray, Bayesian inverse problems with non-conjugate priors, Electronic Journal of Statistics, 7 (2013), 2516-2549. doi: 10.1214/13-EJS851.
[74]	M. Reimherr, Functional regression with repeated eigenvalues, Statistics & Probability Letters, 107 (2015), 62-70. doi: 10.1016/j.spl.2015.07.037.
[75]	M. Reimherr, B. Sriperumbudur and H. B. Kang, Optimal function-on-scalar regression over complex domains, Electronic Journal of Statistics, 17 (2023), 156-197. doi: 10.1214/22-EJS2096.
[76]	L. Scarabosio, Deep neural network surrogates for nonsmooth quantities of interest in shape uncertainty quantification, SIAM/ASA Journal on Uncertainty Quantification, 10 (2022), 975-1011. doi: 10.1137/21M1393078.
[77]	F. Schäfer and H. Owhadi, Sparse recovery of elliptic solvers from matrix-vector products, SIAM Journal on Scientific Computing, 46 (2024), A998-A1025. doi: 10.1137/22M154226X.
[78]	Z. Shen, H. Yang and S. Zhang, Deep network approximation characterized by number of neurons, Communications in Computational Physics, 28 (2020), 1768-1811. doi: 10.4208/cicp.OA-2020-0149.
[79]	Z. Shi, J. Fan, L. Song, D.-X. Zhou and J. A. K. Suykens, Nonlinear functional regression by functional deep neural network with kernel embedding, preprint, arXiv: 2401.02890.
[80]	K. Shukla, V. Oommen, A. Peyvan, M. Penwarden, N. Plewacki, L. Bravo, A. Ghoshal, R. M. Kirby and G. E. Karniadakis, Deep neural operators as accurate surrogates for shape optimization, Engineering Applications of Artificial Intelligence, 129. doi: 10.1016/j.engappai.2023.107615.
[81]	J. L. Steger and D. S. Chaussee, Generation of body-fitted coordinates using hyperbolic partial differential equations, SIAM Journal on Scientific and Statistical Computing, 1 (1980), 431-437. doi: 10.1137/0901031.
[82]	G. Stepaniants, Learning partial differential equations in reproducing kernel Hilbertspaces, Journal of Machine Learning Research, 24 (2023), Paper No. [86], 72 pp.
[83]	M. Takamoto, T. Praditia, R. Leiteritz, D. MacKinlay, F. Alesiani, D. Pflüger and M. Niepert, PDEBench: An extensive benchmark for scientific machine learning, in Advances in Neural Information Processing Systems (eds. S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho and A. Oh), 35 Curran Associates, Inc., 2022, 1596-1611.
[84]	Y. S. Teh, S. Ghosh and K. Bhattacharya, Machine-learned prediction of the electronic fields in a crystal, Mechanics of Materials, 163. doi: 10.1016/j.mechmat.2021.104070.
[85]	M. Trabs, Bayesian inverse problems with unknown operators, Inverse Problems, 34 (2018), 085001, 27 pp. doi: 10.1088/1361-6420/aac3aa.
[86]	R. Vershynin, High-Dimensional Probability: An Introduction with Applications in Data Science, vol. 47, Cambridge University Press, 2018. doi: 10.1017/9781108231596.
[87]	M. J. Wainwright, High-Dimensional Statistics: A Non-Asymptotic Viewpoint, Cambridge University Press, 2019. doi: 10.1017/9781108627771.
[88]	H. You, Q. Zhang, C. J. Ross, C.-H. Lee and Y. Yu, Learning deep implicit Fourier neural operators (IFNOs) with applications to heterogeneous material modeling, Computer Methods in Applied Mechanics and Engineering, 398 (2022), Paper No. 115296, 36 pp. doi: 10.1016/j.cma.2022.115296.
[89]	M. Yuan and T. T. Cai, A reproducing kernel Hilbert space approach to functional linear regression, Annals of Statistics, 38 (2010), 3412-3444.
[90]	F. Zhang, W. Zhang, R. Li and H. Lian, Faster convergence rate for functional linear regression in reproducing kernel Hilbert spaces, Statistics, 54 (2020), 167-181. doi: 10.1080/02331888.2019.1694931.
[91]	Z. Zhang, F. Bao and G. Zhang, Improving the expressive power of deep neural networks through integral activation transform, preprint, arXiv: 2312.12578.
[92]	T. Zhou, X. Wan, D. Z. Huang, Z. Li, Z. Peng, A. Anandkumar, J. F. Brady, P. W. Sternberg and C. Daraio, AI-aided geometric design of anti-infection catheters, Science Advances, 10. doi: 10.1126/sciadv.adj1741.