American Institute of Mathematical Sciences

Advanced
December  2020, 2(4): 443-485. doi: 10.3934/fods.2020021

Estimating linear response statistics using orthogonal polynomials: An RKHS formulation

He Zhang 1,, , John Harlim 2, and  Xiantao Li 1,

1. 

Department of Mathematics, The Pennsylvania State University, University Park, PA 16802, USA
2. 

Department of Mathematics, Department of Meteorology and Atmospheric Science, Institute for Computational and Data Sciences, The Pennsylvania State University, University Park, PA 16802, USA

* Corresponding author

Received  September 2020 Published  December 2020

Fund Project: The research of XL was supported under the NSF grant DMS-1819011 and JH was supported under the NSF grant DMS-1854299

We study the problem of estimating linear response statistics under external perturbations using time series of unperturbed dynamics. Based on the fluctuation-dissipation theory, this problem is reformulated as an unsupervised learning task of estimating a density function. We consider a nonparametric density estimator formulated by the kernel embedding of distributions with "Mercer-type" kernels, constructed based on the classical orthogonal polynomials defined on non-compact domains. While the resulting representation is analogous to Polynomial Chaos Expansion (PCE), the connection to the reproducing kernel Hilbert space (RKHS) theory allows one to establish the uniform convergence of the estimator and to systematically address a practical question of identifying the PCE basis for a consistent estimation. We also provide practical conditions for the well-posedness of not only the estimator but also of the underlying response statistics. Finally, we provide a statistical error bound for the density estimation that accounts for the Monte-Carlo averaging over non-i.i.d time series and the biases due to a finite basis truncation. This error bound provides a means to understand the feasibility as well as limitation of the kernel embedding with Mercer-type kernels. Numerically, we verify the effectiveness of the estimator on two stochastic dynamics with known, yet, non-trivial equilibrium densities.

Keywords: Linear response theory, kernel embedding, orthogonal polynomial, reproducing kernel hilbert space, Mercer's theorem, Mercer-type kernel.
Mathematics Subject Classification: Primary: 46E22, 62G07; Secondary: 82C31, 33C45, 33C50, 37A25.
Citation: He Zhang, John Harlim, Xiantao Li. Estimating linear response statistics using orthogonal polynomials: An RKHS formulation. Foundations of Data Science, 2020, 2 (4) : 443-485. doi: 10.3934/fods.2020021
References:
[1]

M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions, with Formulas, Graphs, and Mathematical Tables, volume 55, Third printing, with corrections. US Government printing office, 1965.  Google Scholar

[2]

W. A. Al-Salam, Operational representations for the Laguerre and other polynomials, Duke Math. J., 31 (1964), 127-142.   Google Scholar

[3]

D. F. Anderson and J. C. Mattingly, A weak trapezoidal method for a class of stochastic differential equations, Commun. Math. Sci., 9 (2011), 301-318.  doi: 10.4310/CMS.2011.v9.n1.a15.  Google Scholar

[4]

M. Baiesi, C. Maes and B. Wynants, Fluctuations and response of nonequilibrium states, Physical review letters, 103 (2009), 010602. doi: 10.1103/PhysRevLett.103.010602.  Google Scholar

[5]

T. Berry and J. Harlim, Correcting biased observation model error in data assimilation, Monthly Weather Review, 145 (2017), 2833-2853.  doi: 10.1175/MWR-D-16-0428.1.  Google Scholar

[6]

M. Branicki and A. J. Majda, Fundamental limitations of polynomial chaos for uncertainty quantification in systems with intermittent instabilities, Commun. Math. Sci., 11 (2013), 55-103.  doi: 10.4310/CMS.2013.v11.n1.a3.  Google Scholar

[7]

Y. Cao and Q. Gu, Generalization error bounds of gradient descent for learning over-parameterized deep ReLU networks, arXiv preprint, arXiv: 1902.01384, 2019. Google Scholar

[8]

L. Carlitz, The product of several Hermite or Laguerre polynomials, Monatsh. Math., 66 (1962), 393-396.  doi: 10.1007/BF01298234.  Google Scholar

[9]

R. R. Coifman and S. Lafon, Diffusion maps, Appl. Comput. Harmon. Anal., 21 (2006), 5-30.  doi: 10.1016/j.acha.2006.04.006.  Google Scholar

[10]

B. Colbois, A. EI Soufi and A. Savo, Eigenvalues of the Laplacian on a compact manifold with density, Comm. Anal. Geom., 23 (2015), 639-670.  doi: 10.4310/CAG.2015.v23.n3.a6.  Google Scholar

[11]

Y. A. Davydov, Convergence of distributions generated by stationary stochastic processes, Theory of Probability & its Applications, 13 (1968), 691-696.   Google Scholar

[12]

O. G. Ernst, A. Mugler, H.-J. Starkloff and E. Ullmann, On the convergence of generalized polynomial chaos expansions, ESAIM Math. Model. Numer. Anal., 46 (2012), 317-339.  doi: 10.1051/m2an/2011045.  Google Scholar

[13]

D. J. Evans and G. P. Morriss, Nonlinear-response theory for steady planar Couette flow, Physical Review A, 30 (1984), 1528. doi: 10.1103/PhysRevA.30.1528.  Google Scholar

[14] D. J. Evans and G. P. Morriss, Statistical Mechanics of Nonequilibrium Liquids, Cambridge University Press, 2008.   Google Scholar
[15]

H. Flyvbjerg and H. G. Petersen, Error estimates on averages of correlated data, J. Chem. Phys., 91 (1989), 461-466.  doi: 10.1063/1.457480.  Google Scholar

[16]

N. García Trillos, M. Gerlach, M. Hein and D. Slepčev, Error estimates for spectral convergence of the graph Laplacian on random geometric graphs toward the Laplace–Beltrami operator, Found. Comput. Math., 20 (2020), 827-887.  doi: 10.1007/s10208-019-09436-w.  Google Scholar

[17]

M. S. Green, Markoff random processes and the statistical mechanics of time-dependent phenomena. ii. irreversible processes in fluids, The Journal of Chemical Physics, 22 (1954), 398-413.  doi: 10.1063/1.1740082.  Google Scholar

[18]

M. Hairer and A. J. Majda, A simple framework to justify linear response theory, Nonlinearity, 23 (2010), 909-922. doi: 10.1088/0951-7715/23/4/008.  Google Scholar

[19]

H. Hang and I. Steinwart, Fast learning from $\alpha$-mixing observations, J. Multivariate Anal., 127 (2014), 184-199.  doi: 10.1016/j.jmva.2014.02.012.  Google Scholar

[20]

A. Hannachi and A. O'Neill, Atmospheric multiple equilibria and non-Gaussian behaviour in model simulations, Quarterly Journal of the Royal Meteorological Society, 127 (2001), 939-958.  doi: 10.1002/qj.49712757312.  Google Scholar

[21]

J.-N. Hwang, S.-R. Lay and A. Lippman, Nonparametric multivariate density estimation: A comparative study, IEEE Transactions on Signal Processing, 42 (1994), 2795-2810.   Google Scholar

[22]

S. W. Jiang and J. Harlim, Parameter estimation with data-driven nonparametric likelihood functions, Entropy, 21 (2019), Paper No. 559, 32 pp. doi: 10.3390/e21060559.  Google Scholar

[23]

S. W. Jiang and J. Harlim, Modeling of missing dynamical systems: Deriving parametric models using a nonparametric framework, Res. Math. Sci., 7 (2020), Paper No. 16, 25 pp. doi: 10.1007/s40687-020-00217-4.  Google Scholar

[24]

M. S. Jolly, I. G. Kevrekidis and E. S. Titi, Approximate inertial manifolds for the Kuramoto-Sivashinsky equation: Analysis and computations, Phys. D, 44 (1990), 38-60.  doi: 10.1016/0167-2789(90)90046-R.  Google Scholar

[25]

R. Kubo, Statistical-mechanical theory of irreversible processes. I. General theory and simple applications to magnetic and conduction problems, J. Phys. Soc. Japan, 12 (1957), 570-586.  doi: 10.1143/JPSJ.12.570.  Google Scholar

[26]

R. Kubo, M. Toda and N. Hashitsume, Statistical Physics II. Nonquilibrium Statistical Mechanics, Springer-Verlag, Berlin, 1985. doi: 10.1007/978-3-642-96701-6.  Google Scholar

[27]

B. Leimkuhler, C. Matthews and G. Stoltz, The computation of averages from equilibrium and nonequilibrium Langevin molecular dynamics, IMA J. Numer. Anal., 36 (2016), 13-79.  doi: 10.1093/imanum/dru056.  Google Scholar

[28]

C. E. Leith, Climate response and fluctuation dissipation, Journal of the Atmospheric Sciences, 32 (1975), 2022-2026.  doi: 10.1175/1520-0469(1975)032<2022:CRAFD>2.0.CO;2.  Google Scholar

[29]

A. L. Levin and D. S. Lubinsky, Christoffel functions, orthogonal polynomials, and Nevai's conjecture for Freud weights, Constr. Approx., 8 (1992), 463-535.  doi: 10.1007/BF01203463.  Google Scholar

[30]

L. Liu, D. Li and W. H. Wong, Convergence rates of a partition based bayesian multivariate density estimation method, In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 4738–4746. Curran Associates, Inc., 2017. Google Scholar

[31]

L. Lu, H. Jiang and W. H. Wong, Multivariate density estimation by Bayesian sequential partitioning, J. Amer. Statist. Assoc., 108 (2013), 1402-1410.  doi: 10.1080/01621459.2013.813389.  Google Scholar

[32]

F. Lu, M. Morzfeld, X. Tu and A. J. Chorin, Limitations of polynomial chaos expansions in the Bayesian solution of inverse problems, J. Comput. Phys., 282 (2015), 138-147.  doi: 10.1016/j.jcp.2014.11.010.  Google Scholar

[33]

A. J. Majda, R. V. Abramov and M. J. Grote, Information Theory and Stochastics for Multiscale Nonlinear Systems, CRM Monograph Series v.25, American Mathematical Society, Providence, Rhode Island, USA, 2005. doi: 10.1090/crmm/025.  Google Scholar

[34]

J. C. Mattingly, A. M. Stuart and D. J. Higham, Ergodicity for SDEs and approximations: Locally Lipschitz vector fields and degenerate noise, Stochastic Process. Appl., 101 (2002), 185-232.  doi: 10.1016/S0304-4149(02)00150-3.  Google Scholar

[35]

F. G. Mehler, Ueber die Entwicklung einer Function von beliebig vielen Variablen nach Laplaceschen Functionen höherer Ordnung, J. Reine Angew. Math., 66 (1866), 161-176.  doi: 10.1515/crll.1866.66.161.  Google Scholar

[36]

K. Muandet, K. Fukumizu, B. Sriperumbudur and B. Schölkopf, Kernel mean embedding of distributions: A review and beyond, Foundations and Trends® in Machine Learning, 10 (2017), 1–141. Google Scholar

[37]

EA Nadaraya, On non-parametric estimates of density functions and regression curves, Theory of Probability & its Applications, 10 (1965), 186-190.   Google Scholar

[38]

G. Papamakarios, T. Pavlakou and I. Murray, Masked autoregressive flow for density estimation, Advances in Neural Information Processing Systems, (2017), 2338–2347. Google Scholar

[39]

G. A. Pavliotis, Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations, volume 60. Springer, 2014. doi: 10.1007/978-1-4939-1323-7.  Google Scholar

[40]

M. Rosenblatt, Remarks on some nonparametric estimates of a density function, Ann. Math. Statist., 27 (1956), 832-837.  doi: 10.1214/aoms/1177728190.  Google Scholar

[41]

B. Roux, The calculation of the potential of mean force using computer simulations, Computer Physics Communications, 91 (1995), 275-282.  doi: 10.1016/0010-4655(95)00053-I.  Google Scholar

[42]

E. F. Schuster, Estimation of a probability density function and its derivatives, Ann. Math. Statist., 40 (1969), 1187-1195.  doi: 10.1214/aoms/1177697495.  Google Scholar

[43]

B. W. Silverman, Density Estimation for Statistics and Data Analysis, volume 26. Monographs on Statistics and Applied Probability. Chapman & Hall, London, 1986.  Google Scholar

[44]

B. Sriperumbudur, K. Fukumizu, A. Gretton, A. Hyvärinen and R. Kumar, Density estimation in infinite dimensional exponential families, J. Mach. Learn. Res., 18 (2017), Paper No. 57, 59 pp.  Google Scholar

[45]

B. K. Sriperumbudur, K. Fukumizu and G. R. G. Lanckriet, On the relation between universality, characteristic kernels and RKHS embedding of measures, In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, (2010), 773–780. Google Scholar

[46]

B. K. Sriperumbudur, K. Fukumizu and G. R. Lanckriet, Universality, characteristic kernels and RKHS embedding of measures, J. Mach. Learn. Res., 12 (2011), 2389-2410.   Google Scholar

[47]

I. Steinwart, On the influence of the kernel on the consistency of support vector machines, J. Mach. Learn. Res., 2 (2001), 67-93.   Google Scholar

[48]

I. Steinwart and A. Christmann, Support Vector Machines, Springer, 2008.  Google Scholar

[49]

C. J. Stone, Optimal global rates of convergence for nonparametric regression, Ann. Statist., 10 (1982), 1040-1053.  doi: 10.1214/aos/1176345969.  Google Scholar

[50]

G. Szegö, Orthogonal Polynomials, volume 23. American Mathematical Soc., 1939.  Google Scholar

[51]

A. Telatovich and X. Li, The strong convergence of operator-splitting methods for the Langevin dynamics model, arXiv preprint, arXiv: 1706.04237, 2017. Google Scholar

[52]

B. Uria, M.-A. Côté, K. Gregor, I. Murray and H. Larochelle, Neural autoregressive distribution estimation, J. Mach. Learn. Res., 17 (2016), Paper No. 205, 37 pp.  Google Scholar

[53]

Z. Wang and D. W. Scott, Nonparametric density estimation for high-dimensional data–Algorithms and applications, Wiley Interdiscip. Rev. Comput. Stat., 11 (2019), e1461, 16 pp. doi: 10.1002/wics.1461.  Google Scholar

[54]

L. Wasserman, All of Nonparametric Statistics, Springer Science & Business Media, 2006.  Google Scholar

[55]

G. N. Watson, Notes on generating functions of polynomials: (1) Laguerre polynomials, J. London Math. Soc., 8 (1933), 189-192.  doi: 10.1112/jlms/s1-8.3.189.  Google Scholar

[56] D. Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach, Princeton University Press, 2010.   Google Scholar
[57]

H. Zhang, X. Li and J. Harlim, A parameter estimation method using linear response statistics: Numerical scheme, Chaos, 29 (2019), 033101, 21 pp. doi: 10.1063/1.5081744.  Google Scholar

[58]

H. Zhang, X. Li and J. Harlim, Linear response based parameter estimation in the presence of model error, arXiv preprint, arXiv: 1910.14113, 2019. Google Scholar

show all references

References:
[1]

M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions, with Formulas, Graphs, and Mathematical Tables, volume 55, Third printing, with corrections. US Government printing office, 1965.  Google Scholar

[2]

W. A. Al-Salam, Operational representations for the Laguerre and other polynomials, Duke Math. J., 31 (1964), 127-142.   Google Scholar

[3]

D. F. Anderson and J. C. Mattingly, A weak trapezoidal method for a class of stochastic differential equations, Commun. Math. Sci., 9 (2011), 301-318.  doi: 10.4310/CMS.2011.v9.n1.a15.  Google Scholar

[4]

M. Baiesi, C. Maes and B. Wynants, Fluctuations and response of nonequilibrium states, Physical review letters, 103 (2009), 010602. doi: 10.1103/PhysRevLett.103.010602.  Google Scholar

[5]

T. Berry and J. Harlim, Correcting biased observation model error in data assimilation, Monthly Weather Review, 145 (2017), 2833-2853.  doi: 10.1175/MWR-D-16-0428.1.  Google Scholar

[6]

M. Branicki and A. J. Majda, Fundamental limitations of polynomial chaos for uncertainty quantification in systems with intermittent instabilities, Commun. Math. Sci., 11 (2013), 55-103.  doi: 10.4310/CMS.2013.v11.n1.a3.  Google Scholar

[7]

Y. Cao and Q. Gu, Generalization error bounds of gradient descent for learning over-parameterized deep ReLU networks, arXiv preprint, arXiv: 1902.01384, 2019. Google Scholar

[8]

L. Carlitz, The product of several Hermite or Laguerre polynomials, Monatsh. Math., 66 (1962), 393-396.  doi: 10.1007/BF01298234.  Google Scholar

[9]

R. R. Coifman and S. Lafon, Diffusion maps, Appl. Comput. Harmon. Anal., 21 (2006), 5-30.  doi: 10.1016/j.acha.2006.04.006.  Google Scholar

[10]

B. Colbois, A. EI Soufi and A. Savo, Eigenvalues of the Laplacian on a compact manifold with density, Comm. Anal. Geom., 23 (2015), 639-670.  doi: 10.4310/CAG.2015.v23.n3.a6.  Google Scholar

[11]

Y. A. Davydov, Convergence of distributions generated by stationary stochastic processes, Theory of Probability & its Applications, 13 (1968), 691-696.   Google Scholar

[12]

O. G. Ernst, A. Mugler, H.-J. Starkloff and E. Ullmann, On the convergence of generalized polynomial chaos expansions, ESAIM Math. Model. Numer. Anal., 46 (2012), 317-339.  doi: 10.1051/m2an/2011045.  Google Scholar

[13]

D. J. Evans and G. P. Morriss, Nonlinear-response theory for steady planar Couette flow, Physical Review A, 30 (1984), 1528. doi: 10.1103/PhysRevA.30.1528.  Google Scholar

[14] D. J. Evans and G. P. Morriss, Statistical Mechanics of Nonequilibrium Liquids, Cambridge University Press, 2008.   Google Scholar
[15]

H. Flyvbjerg and H. G. Petersen, Error estimates on averages of correlated data, J. Chem. Phys., 91 (1989), 461-466.  doi: 10.1063/1.457480.  Google Scholar

[16]

N. García Trillos, M. Gerlach, M. Hein and D. Slepčev, Error estimates for spectral convergence of the graph Laplacian on random geometric graphs toward the Laplace–Beltrami operator, Found. Comput. Math., 20 (2020), 827-887.  doi: 10.1007/s10208-019-09436-w.  Google Scholar

[17]

M. S. Green, Markoff random processes and the statistical mechanics of time-dependent phenomena. ii. irreversible processes in fluids, The Journal of Chemical Physics, 22 (1954), 398-413.  doi: 10.1063/1.1740082.  Google Scholar

[18]

M. Hairer and A. J. Majda, A simple framework to justify linear response theory, Nonlinearity, 23 (2010), 909-922. doi: 10.1088/0951-7715/23/4/008.  Google Scholar

[19]

H. Hang and I. Steinwart, Fast learning from $\alpha$-mixing observations, J. Multivariate Anal., 127 (2014), 184-199.  doi: 10.1016/j.jmva.2014.02.012.  Google Scholar

[20]

A. Hannachi and A. O'Neill, Atmospheric multiple equilibria and non-Gaussian behaviour in model simulations, Quarterly Journal of the Royal Meteorological Society, 127 (2001), 939-958.  doi: 10.1002/qj.49712757312.  Google Scholar

[21]

J.-N. Hwang, S.-R. Lay and A. Lippman, Nonparametric multivariate density estimation: A comparative study, IEEE Transactions on Signal Processing, 42 (1994), 2795-2810.   Google Scholar

[22]

S. W. Jiang and J. Harlim, Parameter estimation with data-driven nonparametric likelihood functions, Entropy, 21 (2019), Paper No. 559, 32 pp. doi: 10.3390/e21060559.  Google Scholar

[23]

S. W. Jiang and J. Harlim, Modeling of missing dynamical systems: Deriving parametric models using a nonparametric framework, Res. Math. Sci., 7 (2020), Paper No. 16, 25 pp. doi: 10.1007/s40687-020-00217-4.  Google Scholar

[24]

M. S. Jolly, I. G. Kevrekidis and E. S. Titi, Approximate inertial manifolds for the Kuramoto-Sivashinsky equation: Analysis and computations, Phys. D, 44 (1990), 38-60.  doi: 10.1016/0167-2789(90)90046-R.  Google Scholar

[25]

R. Kubo, Statistical-mechanical theory of irreversible processes. I. General theory and simple applications to magnetic and conduction problems, J. Phys. Soc. Japan, 12 (1957), 570-586.  doi: 10.1143/JPSJ.12.570.  Google Scholar

[26]

R. Kubo, M. Toda and N. Hashitsume, Statistical Physics II. Nonquilibrium Statistical Mechanics, Springer-Verlag, Berlin, 1985. doi: 10.1007/978-3-642-96701-6.  Google Scholar

[27]

B. Leimkuhler, C. Matthews and G. Stoltz, The computation of averages from equilibrium and nonequilibrium Langevin molecular dynamics, IMA J. Numer. Anal., 36 (2016), 13-79.  doi: 10.1093/imanum/dru056.  Google Scholar

[28]

C. E. Leith, Climate response and fluctuation dissipation, Journal of the Atmospheric Sciences, 32 (1975), 2022-2026.  doi: 10.1175/1520-0469(1975)032<2022:CRAFD>2.0.CO;2.  Google Scholar

[29]

A. L. Levin and D. S. Lubinsky, Christoffel functions, orthogonal polynomials, and Nevai's conjecture for Freud weights, Constr. Approx., 8 (1992), 463-535.  doi: 10.1007/BF01203463.  Google Scholar

[30]

L. Liu, D. Li and W. H. Wong, Convergence rates of a partition based bayesian multivariate density estimation method, In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 4738–4746. Curran Associates, Inc., 2017. Google Scholar

[31]

L. Lu, H. Jiang and W. H. Wong, Multivariate density estimation by Bayesian sequential partitioning, J. Amer. Statist. Assoc., 108 (2013), 1402-1410.  doi: 10.1080/01621459.2013.813389.  Google Scholar

[32]

F. Lu, M. Morzfeld, X. Tu and A. J. Chorin, Limitations of polynomial chaos expansions in the Bayesian solution of inverse problems, J. Comput. Phys., 282 (2015), 138-147.  doi: 10.1016/j.jcp.2014.11.010.  Google Scholar

[33]

A. J. Majda, R. V. Abramov and M. J. Grote, Information Theory and Stochastics for Multiscale Nonlinear Systems, CRM Monograph Series v.25, American Mathematical Society, Providence, Rhode Island, USA, 2005. doi: 10.1090/crmm/025.  Google Scholar

[34]

J. C. Mattingly, A. M. Stuart and D. J. Higham, Ergodicity for SDEs and approximations: Locally Lipschitz vector fields and degenerate noise, Stochastic Process. Appl., 101 (2002), 185-232.  doi: 10.1016/S0304-4149(02)00150-3.  Google Scholar

[35]

F. G. Mehler, Ueber die Entwicklung einer Function von beliebig vielen Variablen nach Laplaceschen Functionen höherer Ordnung, J. Reine Angew. Math., 66 (1866), 161-176.  doi: 10.1515/crll.1866.66.161.  Google Scholar

[36]

K. Muandet, K. Fukumizu, B. Sriperumbudur and B. Schölkopf, Kernel mean embedding of distributions: A review and beyond, Foundations and Trends® in Machine Learning, 10 (2017), 1–141. Google Scholar

[37]

EA Nadaraya, On non-parametric estimates of density functions and regression curves, Theory of Probability & its Applications, 10 (1965), 186-190.   Google Scholar

[38]

G. Papamakarios, T. Pavlakou and I. Murray, Masked autoregressive flow for density estimation, Advances in Neural Information Processing Systems, (2017), 2338–2347. Google Scholar

[39]

G. A. Pavliotis, Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations, volume 60. Springer, 2014. doi: 10.1007/978-1-4939-1323-7.  Google Scholar

[40]

M. Rosenblatt, Remarks on some nonparametric estimates of a density function, Ann. Math. Statist., 27 (1956), 832-837.  doi: 10.1214/aoms/1177728190.  Google Scholar

[41]

B. Roux, The calculation of the potential of mean force using computer simulations, Computer Physics Communications, 91 (1995), 275-282.  doi: 10.1016/0010-4655(95)00053-I.  Google Scholar

[42]

E. F. Schuster, Estimation of a probability density function and its derivatives, Ann. Math. Statist., 40 (1969), 1187-1195.  doi: 10.1214/aoms/1177697495.  Google Scholar

[43]

B. W. Silverman, Density Estimation for Statistics and Data Analysis, volume 26. Monographs on Statistics and Applied Probability. Chapman & Hall, London, 1986.  Google Scholar

[44]

B. Sriperumbudur, K. Fukumizu, A. Gretton, A. Hyvärinen and R. Kumar, Density estimation in infinite dimensional exponential families, J. Mach. Learn. Res., 18 (2017), Paper No. 57, 59 pp.  Google Scholar

[45]

B. K. Sriperumbudur, K. Fukumizu and G. R. G. Lanckriet, On the relation between universality, characteristic kernels and RKHS embedding of measures, In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, (2010), 773–780. Google Scholar

[46]

B. K. Sriperumbudur, K. Fukumizu and G. R. Lanckriet, Universality, characteristic kernels and RKHS embedding of measures, J. Mach. Learn. Res., 12 (2011), 2389-2410.   Google Scholar

[47]

I. Steinwart, On the influence of the kernel on the consistency of support vector machines, J. Mach. Learn. Res., 2 (2001), 67-93.   Google Scholar

[48]

I. Steinwart and A. Christmann, Support Vector Machines, Springer, 2008.  Google Scholar

[49]

C. J. Stone, Optimal global rates of convergence for nonparametric regression, Ann. Statist., 10 (1982), 1040-1053.  doi: 10.1214/aos/1176345969.  Google Scholar

[50]

G. Szegö, Orthogonal Polynomials, volume 23. American Mathematical Soc., 1939.  Google Scholar

[51]

A. Telatovich and X. Li, The strong convergence of operator-splitting methods for the Langevin dynamics model, arXiv preprint, arXiv: 1706.04237, 2017. Google Scholar

[52]

B. Uria, M.-A. Côté, K. Gregor, I. Murray and H. Larochelle, Neural autoregressive distribution estimation, J. Mach. Learn. Res., 17 (2016), Paper No. 205, 37 pp.  Google Scholar

[53]

Z. Wang and D. W. Scott, Nonparametric density estimation for high-dimensional data–Algorithms and applications, Wiley Interdiscip. Rev. Comput. Stat., 11 (2019), e1461, 16 pp. doi: 10.1002/wics.1461.  Google Scholar

[54]

L. Wasserman, All of Nonparametric Statistics, Springer Science & Business Media, 2006.  Google Scholar

[55]

G. N. Watson, Notes on generating functions of polynomials: (1) Laguerre polynomials, J. London Math. Soc., 8 (1933), 189-192.  doi: 10.1112/jlms/s1-8.3.189.  Google Scholar

[56] D. Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach, Princeton University Press, 2010.   Google Scholar
[57]

H. Zhang, X. Li and J. Harlim, A parameter estimation method using linear response statistics: Numerical scheme, Chaos, 29 (2019), 033101, 21 pp. doi: 10.1063/1.5081744.  Google Scholar

[58]

H. Zhang, X. Li and J. Harlim, Linear response based parameter estimation in the presence of model error, arXiv preprint, arXiv: 1910.14113, 2019. Google Scholar

Figure 1.  On the left panel, we show an example of how $ \eta_M $ in (54) behaves as a function of $ M $. On the same panel, we also show the rejection probability $ \mathcal{R}_M $ as a function of $ M $. One can see that as the algorithm converges (with small $ \eta_M $), the rejection probability converges to a relatively small value. On the right panel, we show $ \delta_M $ as a function of $ M $. Here, there is no pattern for $ \delta_M $. In practice, since $ \delta_M $ can be arbitrarily small, one can set $ \delta $ in (43) to be slightly larger than the floating-point single precision to guarantee a well-posed estimator. The results in this figure are based on the gradient system to be discussed in Section 5.1
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  The equilibrium distribution of the triple-well model (59) (upper left panel) and its kernel embedding estimate (upper right panel) based on a total of $ 1\times 10^{7} $ sample. The contour plot (lower left panel) displays the absolute error of the estimate. The error plot (lower right panel) shows the $ \ell_{\infty} $-error of the estimates $ \hat{k}_{A} $ via kernel embedding linear response. We separate the diagonal entries (D) from the non-diagonal entries (ND) due to their scale difference
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  The linear response operator $ k_{A} $ in (62) (blue solid) and the corresponding estimates $ \hat{k}_{A} $ in (63) via kernel embedding linear response (red dash) and KDE (yellow dot-dash). For the two-point statistics, both $ k_{A} $ and $ \hat{k}_{A} $ are computed via Monte-Carlo. The diagonal entries of $ k_{A} $ and $ \hat{k}_{A} $ are normalized so that they share the same initial value $ 1 $. Two insert figures are added to the diagonal entries to show the details of the estimates at the initial stage
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  Left panel: $ \eta_M $ for the Gaussian marginal density of variable $ v $ as a function of $ M = M_2 $(dotted blue line). In the same panel, we also show $ \eta_M $ for the marginal density of variable $ x $ (dashes blue) and the rejection probability $ \mathcal{R}_M $ (solid red) as functions of $ M = M_1 $ for a fixed $ M_2 = 0 $. Right panel: The marginal distribution of $ x $ (left) of the Langevin dynamics (64) at equilibrium. The kernel embedding estimate uses Laguerre polynomials with $ M = 90 $. All the results in this picture are based on a total of $ N = 10^{7} $ samples
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3, the diagonal entries of $ k_{A} $ and $ \hat{k}_{A} $ are normalized so that they share the same initial value $ 1 $. The $ (1, 2) $ and $ (2, 2) $ components reach perfect fits for both methods since $ v $ is Gaussian at the equilibrium">Figure 5.  The linear response operator $ k_{A} $ in (66) (blue solid) and the corresponding estimates $ \hat{k}_{A} $ in (68) via kernel embedding linear response (red dash) and KDE (yellow dot-dash). All the statistics are computed via Monte-Carlo. Similar to Figure 3, the diagonal entries of $ k_{A} $ and $ \hat{k}_{A} $ are normalized so that they share the same initial value $ 1 $. The $ (1, 2) $ and $ (2, 2) $ components reach perfect fits for both methods since $ v $ is Gaussian at the equilibrium
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  Graph of $ k_{\beta, 0.64, 1}(x, x) $ in (32) for $ \beta = 0.42 $ (blue-solid), $ \beta = 0.45 $ (red-dot-dash), and $ \beta = 0.48 $ (yellow-dash). Notice that for $ \rho = 0.64 $, to ensure the boundedness, we need $ \beta \geq \frac{0.8}{1+0.8}\approx 0.44 $, which is consistent with the numerical results
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  Elapsed time (based on a desktop computer, equipped with a 3.2GHz quad-core Intel Core i5 processor with 32Gb RAM) of the KDE approach and the kernel embedding approach in computing the linear response statistics
Method Number of Basis Elapsed Time (s)
KDE (Triple-Well, $ N = 1\times 10^7 $) $ 1\times 10^7 $ $ 1.99 \times 10^4 $
Kernel Embedding (Triple-Well, $ M = 60 $) $ 1891 $ $ 1.54 \times 10^3 $
Kernel Embedding (Langevin, $ M_1 = 90 $, $ M_2 = 0 $) $ 91 $ $ 8.21 $
Table Options

Download as excel

Method Number of Basis Elapsed Time (s)
KDE (Triple-Well, $ N = 1\times 10^7 $) $ 1\times 10^7 $ $ 1.99 \times 10^4 $
Kernel Embedding (Triple-Well, $ M = 60 $) $ 1891 $ $ 1.54 \times 10^3 $
Kernel Embedding (Langevin, $ M_1 = 90 $, $ M_2 = 0 $) $ 91 $ $ 8.21 $
[1]

Kaitlyn (Voccola) Muller. A reproducing kernel Hilbert space framework for inverse scattering problems within the Born approximation. Inverse Problems & Imaging, 2019, 13 (6) : 1327-1348. doi: 10.3934/ipi.2019058
[2]

Ali Akgül, Mustafa Inc, Esra Karatas. Reproducing kernel functions for difference equations. Discrete & Continuous Dynamical Systems - S, 2015, 8 (6) : 1055-1064. doi: 10.3934/dcdss.2015.8.1055
[3]

Ali Akgül. A new application of the reproducing kernel method. Discrete & Continuous Dynamical Systems - S, 2021, 14 (7) : 2041-2053. doi: 10.3934/dcdss.2020261
[4]

Liam Burrows, Weihong Guo, Ke Chen, Francesco Torella. Reproducible kernel Hilbert space based global and local image segmentation. Inverse Problems & Imaging, 2021, 15 (1) : 1-25. doi: 10.3934/ipi.2020048
[5]

Xin Guo, Lexin Li, Qiang Wu. Modeling interactive components by coordinate kernel polynomial models. Mathematical Foundations of Computing, 2020, 3 (4) : 263-277. doi: 10.3934/mfc.2020010
[6]

Ying Lin, Rongrong Lin, Qi Ye. Sparse regularized learning in the reproducing kernel banach spaces with the $ \ell^1 $ norm. Mathematical Foundations of Computing, 2020, 3 (3) : 205-218. doi: 10.3934/mfc.2020020
[7]

Steven G. Krantz and Marco M. Peloso. New results on the Bergman kernel of the worm domain in complex space. Electronic Research Announcements, 2007, 14: 35-41. doi: 10.3934/era.2007.14.35
[8]

Philippe Laurençot, Barbara Niethammer, Juan J.L. Velázquez. Oscillatory dynamics in Smoluchowski's coagulation equation with diagonal kernel. Kinetic & Related Models, 2018, 11 (4) : 933-952. doi: 10.3934/krm.2018037
[9]

Shui-Nee Chow, Ke Yin, Hao-Min Zhou, Ali Behrooz. Solving inverse source problems by the Orthogonal Solution and Kernel Correction Algorithm (OSKCA) with applications in fluorescence tomography. Inverse Problems & Imaging, 2014, 8 (1) : 79-102. doi: 10.3934/ipi.2014.8.79
[10]

Fırat Evirgen, Sümeyra Uçar, Necati Özdemir, Zakia Hammouch. System response of an alcoholism model under the effect of immigration via non-singular kernel derivative. Discrete & Continuous Dynamical Systems - S, 2021, 14 (7) : 2199-2212. doi: 10.3934/dcdss.2020145
[11]

Yanqin Bai, Pengfei Ma, Jing Zhang. A polynomial-time interior-point method for circular cone programming based on kernel functions. Journal of Industrial & Management Optimization, 2016, 12 (2) : 739-756. doi: 10.3934/jimo.2016.12.739
[12]

Marjan Uddin, Hazrat Ali. Space-time kernel based numerical method for generalized Black-Scholes equation. Discrete & Continuous Dynamical Systems - S, 2020, 13 (10) : 2905-2915. doi: 10.3934/dcdss.2020221
[13]

Siqi Li, Weiyi Qian. Analysis of complexity of primal-dual interior-point algorithms based on a new kernel function for linear optimization. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 37-46. doi: 10.3934/naco.2015.5.37
[14]

Alfredo Lorenzi, Eugenio Sinestrari. Identifying a BV-kernel in a hyperbolic integrodifferential equation. Discrete & Continuous Dynamical Systems, 2008, 21 (4) : 1199-1219. doi: 10.3934/dcds.2008.21.1199
[15]

Xiao-Qiang Zhao, Shengfan Zhou. Kernel sections for processes and nonautonomous lattice systems. Discrete & Continuous Dynamical Systems - B, 2008, 9 (3&4, May) : 763-785. doi: 10.3934/dcdsb.2008.9.763
[16]

François Bolley, Arnaud Guillin, Xinyu Wang. Non ultracontractive heat kernel bounds by Lyapunov conditions. Discrete & Continuous Dynamical Systems, 2015, 35 (3) : 857-870. doi: 10.3934/dcds.2015.35.857
[17]

Sandra Carillo, Vanda Valente, Giorgio Vergara Caffarelli. Heat conduction with memory: A singular kernel problem. Evolution Equations & Control Theory, 2014, 3 (3) : 399-410. doi: 10.3934/eect.2014.3.399
[18]

Ye Tian, Wei Yang, Gene Lai, Menghan Zhao. Predicting non-life insurer's insolvency using non-kernel fuzzy quadratic surface support vector machines. Journal of Industrial & Management Optimization, 2019, 15 (2) : 985-999. doi: 10.3934/jimo.2018081
[19]

Giorgio Metafune, Chiara Spina. Heat Kernel estimates for some elliptic operators with unbounded diffusion coefficients. Discrete & Continuous Dynamical Systems, 2012, 32 (6) : 2285-2299. doi: 10.3934/dcds.2012.32.2285
[20]

Fabrizio Colombo, Davide Guidetti. Identification of the memory kernel in the strongly damped wave equation by a flux condition. Communications on Pure & Applied Analysis, 2009, 8 (2) : 601-620. doi: 10.3934/cpaa.2009.8.601

 Impact Factor: 

Tools

Metrics

  • PDF downloads (86)
  • HTML views (222)
  • Cited by (1)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences