Harmonic analysis of network systems via kernels and their boundary realizations

Palle Jorgensen; James Tian

doi:10.3934/dcdss.2021105

Article Contents

2023, Volume 16, Issue 2: 277-308. Doi: 10.3934/dcdss.2021105

This issue Previous Article Physics informed model error for data assimilation Next Article Gaussian mixture models for clustering and calibration of ensemble weather forecasts

Harmonic analysis of network systems via kernels and their boundary realizations

Palle Jorgensen^1, and
James Tian^2, ,

1.
Department of Mathematics, The University of Iowa, Iowa City, IA 52242-1419, USA
2.
Mathematical Reviews, 416 4th Street, Ann Arbor, MI 48103-4816, USA

^* Corresponding author: James Tian
^* Corresponding author: James Tian

Received: March 2021

Revised: June 2021

Early access: September 2021

Published: February 2023

Abstract / Introduction Full Text(HTML) Figure(6) Related Papers Cited by

Abstract

With view to applications to harmonic and stochastic analysis of infinite network/graph models, we introduce new tools for realizations and transforms of positive definite kernels (p.d.) $ K $ and their associated reproducing kernel Hilbert spaces. With this we establish two kinds of factorizations: (i) Probabilistic: Starting with a positive definite kernel $ K $ we analyze associated Gaussian processes $ V $. Properties of the Gaussian processes will be derived from certain factorizations of $ K $, arising as a covariance kernel of $ V $. (ii) Geometric analysis: We discuss families of measure spaces arising as boundaries for $ K $. Our results entail an analysis of a partial order on families of p.d. kernels, a duality for operators and frames, optimization, Karhunen–Loève expansions, and factorizations. Applications include a new boundary analysis for the Drury-Arveson kernel, and for certain fractals arising as iterated function systems; and an identification of optimal feature spaces in machine learning models.

Keywords:

Mathematics Subject Classification: Primary: 46N30, 46N50, 42C15; Secondary: 46N20, 31A15, 81S25.

Citation:

Full Text(HTML)

Figure 1. Current flows in a connected resistance network

Download: Full-size image PowerPoint slide

Figure 2. Transition probabilities $ p_{xy} $ at a vertex $ x $ $ \left(\mbox{in }V\right) $

Download: Full-size image PowerPoint slide

Figure 3. $ v_{x}\left(\cdot\right) = \cdot\wedge x $

Download: Full-size image PowerPoint slide

Figure 4. ${}^{1}\!\!\diagup\!\!{}_{4}\; $-Cantor set

Download: Full-size image PowerPoint slide

Figure 5. A Swiss role

Download: Full-size image PowerPoint slide

Figure 6. SVM using Gaussian kernel

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	D. Alpay and V. Bolotnikov, On tangential interpolation in reproducing kernel Hilbert modules and applications, in Topics in Interpolation Theory (Leipzig, 1994), vol. 95 of Oper. Theory Adv. Appl., Birkhäuser, Basel, 1997, 37–68.
[2]	D. Alpay, V. Bolotnikov and H. T. Kaptanoğlu, The Schur algorithm and reproducing kernel Hilbert spaces in the ball, Linear Algebra Appl., 342 (2002), 163-186. doi: 10.1016/S0024-3795(01)00448-7.
[3]	D. Alpay, P. Cerejeiras and U. Kähler, Gleason's problem associated to the fractional Cauchy-Riemann operator, Fueter series, Drury-Arveson space and related topics, Proc. Amer. Math. Soc., 145 (2017), 4821-4835. doi: 10.1090/proc/13613.
[4]	D. Alpay, P. Dewilde and H. Dym, Lossless inverse scattering and reproducing kernels for upper triangular operators, in Extension and Interpolation of Linear Operators and Matrix Functions, vol. 47 of Oper. Theory Adv. Appl., Birkhäuser, Basel, (1990), 61–135.
[5]	D. Alpay, A. Dijksma, J. Rovnyak and H. S. V. de Snoo, Realization and factorization in reproducing kernel Pontryagin spaces, in Operator Theory, System Theory and Related Topics (Beer-Sheva/Rehovot, 1997), vol. 123 of Oper. Theory Adv. Appl., Birkhäuser, Basel, (2001), 43–65.
[6]	D. Alpay and C. Dubi, Some remarks on the smoothing problem in a reproducing kernel Hilbert space, J. Anal. Appl., 4 (2006), 119-132.
[7]	D. Alpay and H. Dym, On reproducing kernel spaces, the Schur algorithm, and interpolation in a general class of domains, in Operator Theory and Complex Analysis (Sapporo, 1991), vol. 59 of Oper. Theory Adv. Appl., Birkhäuser, Basel, 1992, 30–77.
[8]	D. Alpay and H. Dym, On a new class of structured reproducing kernel spaces, J. Funct. Anal., 111 (1993), 1-28. doi: 10.1006/jfan.1993.1001.
[9]	D. Alpay and P. Jorgensen, Spectral theory for Gaussian processes: Reproducing kernels, boundaries, and $L^2$-wavelet generators with fractional scales, Numer. Funct. Anal. Optim., 36 (2015), 1239-1285. doi: 10.1080/01630563.2015.1062777.
[10]	D. Alpay, P. Jorgensen and D. Levanony, On the equivalence of probability spaces, J. Theoret. Probab., 30 (2017), 813-841. doi: 10.1007/s10959-016-0667-7.
[11]	D. Alpay and P. E. T. Jorgensen, Stochastic processes induced by singular operators, Numer. Funct. Anal. Optim., 33 (2012), 708-735. doi: 10.1080/01630563.2012.682132.
[12]	D. Alpay and D. Levanony, On the reproducing kernel Hilbert spaces associated with the fractional and bi-fractional Brownian motions, Potential Anal., 28 (2008), 163-184. doi: 10.1007/s11118-007-9070-4.
[13]	D. Alpay and T. M. Mills, A family of Hilbert spaces which are not reproducing kernel Hilbert spaces, J. Anal. Appl., 1 (2003), 107-111.
[14]	N. Arcozzi and M. Levi, On a class of shift-invariant subspaces of the Drury-Arveson space, Concr. Oper., 5 (2018), 1-8. doi: 10.1515/conop-2018-0001.
[15]	N. Aronszajn, La théorie des noyaux reproduisants et ses applications. I, Proc. Cambridge Philos. Soc., 39 (1943), 133-153. doi: 10.1017/S0305004100017813.
[16]	N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc., 68 (1950), 337-404. doi: 10.1090/S0002-9947-1950-0051437-7.
[17]	N. Aronszajn and K. T. Smith, Characterization of positive reproducing kernels. Applications to Green's functions, Amer. J. Math., 79 (1957), 611-622. doi: 10.2307/2372565.
[18]	W. Arveson, Subalgebras of $C^$-algebras. III. Multivariable operator theory, Acta Math., 181* (1998), 159-228. doi: 10.1007/BF02392585.
[19]	G. Cheng, X. Hou and C. Liu, The singular integral operator induced by Drury-Arveson kernel, Complex Anal. Oper. Theory, 12 (2018), 917-929. doi: 10.1007/s11785-016-0537-4.
[20]	F. Cucker and S. Smale, On the mathematical foundations of learning, Bull. Amer. Math. Soc. (N.S.), 39 (2002), 1–49 (electronic). doi: 10.1090/S0273-0979-01-00923-5.
[21]	S. W. Drury, A generalization of von Neumann's inequality to the complex ball, Proc. Amer. Math. Soc., 68 (1978), 300-304. doi: 10.2307/2043109.
[22]	D. E. Dutkay and P. E. T. Jorgensen, Affine fractals as boundaries and their harmonic analysis, Proc. Amer. Math. Soc., 139 (2011), 3291-3305. doi: 10.1090/S0002-9939-2011-10752-4.
[23]	D. E. Dutkay and P. E. T. Jorgensen, Unitary groups and spectral sets, J. Funct. Anal., 268 (2015), 2102-2141. doi: 10.1016/j.jfa.2015.01.018.
[24]	W. E and S. Wojtowytsch, Kolmogorov width decay and poor approximators in machine learning: Shallow neural networks, random feature models and neural tangent kernels, Res. Math. Sci., 8 (2021), Paper No. 5, 28 pp. doi: 10.1007/s40687-020-00233-4.
[25]	M. Geiger, A. Jacot, S. Spigler, F. Gabriel, L. Sagun, S. d'Ascoli, G. Biroli, C. Hongler and M. Wyart, Scaling description of generalization with number of parameters in deep learning, J. Stat. Mech. Theory Exp., (2020), 023401, 23 pp. doi: 10.1088/1742-5468/ab633c.
[26]	M. Geiger, S. Spigler, A. Jacot and M. Wyart, Disentangling feature and lazy training in deep neural networks, J. Stat. Mech. Theory Exp., (2020), 113301, 27 pp. doi: 10.1088/1742-5468/abc4de.
[27]	Y. Hao, P. Li and K. Zhao, Regularities of semigroups, Carleson measures and the characterizations of BMO-type spaces associated with generalized Schrödinger operators, Banach J. Math. Anal., 13 (2019), 1-25. doi: 10.1215/17358787-2018-0013.
[28]	J. E. Herr, P. E. T. Jorgensen and E. S. Weber, A matrix characterization of boundary representations of positive matrices in the Hardy space, in Frames and Harmonic Analysis, vol. 706 of Contemp. Math., Amer. Math. Soc., Providence, RI, (2018), 255–270. doi: 10.1090/conm/706/14211.
[29]	J. E. Herr, P. E. T. Jorgensen and E. S. Weber, A characterization of boundary representations of positive matrices in the Hardy space via the Abel product, Linear Algebra Appl., 576 (2019), 51-66. doi: 10.1016/j.laa.2018.02.023.
[30]	T. Hida, Quadratic functionals of Brownian motion, J. Multivariate Anal., 1 (1971), 58-69. doi: 10.1016/0047-259X(71)90029-7.
[31]	T. Hida, Brownian Motion, vol. 11 of Applications of Mathematics, Springer-Verlag, New York-Berlin, 1980, Translated from the Japanese by the author and T. P. Speed.
[32]	T. Hida, Stochastic variational calculus, in Stochastic Partial Differential Equations and their Applications (Charlotte, NC, 1991), vol. 176 of Lect. Notes Control Inf. Sci., Springer, Berlin, (1992), 123–134. doi: 10.1007/BFb0007327.
[33]	T. Hofmann, B. Schölkopf and A. J. Smola, Kernel methods in machine learning, Ann. Statist., 36 (2008), 1171-1220. doi: 10.1214/009053607000000677.
[34]	J. B. Hough, M. Krishnapur, Y. Peres and B. Virág, Zeros of Gaussian Analytic Functions and Determinantal Point Processes, vol. 51 of University Lecture Series, American Mathematical Society, Providence, RI, 2009. doi: 10.1090/ulect/051.
[35]	T. Hsing and R. Eubank, Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators, Wiley Series in Probability and Statistics, John Wiley & Sons, Ltd., Chichester, 2015. doi: 10.1002/9781118762547.
[36]	K. Itô, Stochastic Processes, Springer-Verlag, Berlin, 2004, Lectures given at Aarhus University, Reprint of the 1969 original, Edited and with a foreword by Ole E. Barndorff-Nielsen and Ken-iti Sato. doi: 10.1007/978-3-662-10065-3.
[37]	K. Itô and H. P. McKean Jr., Diffusion Processes and their Sample Paths, Die Grundlehren der Mathematischen Wissenschaften, Band 125, Academic Press Inc., Publishers, New York, 1965.
[38]	P. Jorgensen and F. Tian, Transfer operators, induced probability spaces, and random walk models, Markov Process. Related Fields, 23 (2017), 187-210.
[39]	P. E. T. Jorgensen and S. Pedersen, Harmonic analysis of fractal measures, Constr. Approx., 12 (1996), 1-30. doi: 10.1007/BF02432853.
[40]	P. Jorgensen, E. Pearse and F. Tian, Unbounded operators in Hilbert space, duality rules, characteristic projections, and their applications, Anal. Math. Phys., 8 (2018), 351-382. doi: 10.1007/s13324-017-0173-9.
[41]	P. Jorgensen and F. Tian, Discrete reproducing kernel Hilbert spaces: Sampling and distribution of Dirac-masses, J. Mach. Learn. Res., 16 (2015), 3079-3114.
[42]	P. Jorgensen and F. Tian, Positive definite kernels and boundary spaces, Adv. Oper. Theory, 1 (2016), 123-133. doi: 10.22034/aot.1610.1044.
[43]	P. Jorgensen and F. Tian, Generalized Gramians: Creating frame vectors in maximal subspaces, Anal. Appl. (Singap.), 15 (2017), 123-135. doi: 10.1142/S0219530516500019.
[44]	P. Jorgensen and F. Tian, Metric duality between positive definite kernels and boundary processes, Int. J. Appl. Comput. Math., 4 (2018), Paper No. 3, 13 pp. doi: 10.1007/s40819-017-0434-1.
[45]	P. Jorgensen and F. Tian, Non-Commutative Analysis, Hackensack, NJ: World Scientific, 2017. doi: 10.1142/10317.
[46]	P. Jorgensen and F. Tian, Realizations and Factorizations of Positive Definite Kernels, J. Theoret. Probab., 32 (2019), 1925-1942. doi: 10.1007/s10959-018-0868-3.
[47]	P. E. T. Jorgensen, Analysis and Probability: Wavelets, Signals, Fractals, vol. 234 of Graduate Texts in Mathematics, Springer, New York, 2006.
[48]	P. E. T. Jorgensen, Harmonic Analysis: Smooth and Non-Smooth, CBMS Regional Conference Series in Mathematics, Conference Board of the Mathematical Sciences, 2018, https://books.google.com/books?id=lGdxuwEACAAJ. doi: 10.1090/cbms/128.
[49]	P. E. T. Jorgensen and E. P. J. Pearse, Resistance boundaries of infinite networks, in Random Walks, Boundaries and Spectra, vol. 64 of Progr. Probab., Birkhäuser/Springer Basel AG, Basel, 2011,111–142. doi: 10.1007/978-3-0346-0244-0_7.
[50]	P. E. T. Jorgensen and E. P. J. Pearse, Symmetric pairs of unbounded operators in Hilbert space, and their applications in mathematical physics, Math. Phys. Anal. Geom., 20 (2017), Art. 14, 24 pp. doi: 10.1007/s11040-017-9245-1.
[51]	P. E. T. Jorgensen and E. P. J. Pearse, Continuum versus discrete networks, graph Laplacians, and reproducing kernel Hilbert spaces, J. Math. Anal. Appl., 469 (2019), 765-807. doi: 10.1016/j.jmaa.2018.09.035.
[52]	P. E. T. Jorgensen and E. P. J. Pearse, A Hilbert space approach to effective resistance metric, Complex Anal. Oper. Theory, 4 (2010), 975-1013. doi: 10.1007/s11785-009-0041-1.
[53]	P. E. T. Jorgensen and S. Pedersen, Spectral theory for Borel sets in ${\mathbf{R}}^n$ of finite measure, J. Funct. Anal., 107 (1992), 72-104. doi: 10.1016/0022-1236(92)90101-N.
[54]	P. E. T. Jorgensen and S. Pedersen, Harmonic analysis and fractal limit-measures induced by representations of a certain $C^$-algebra, J. Funct. Anal., 125* (1994), 90-110. doi: 10.1006/jfan.1994.1118.
[55]	P. E. T. Jorgensen and S. Pedersen, Dense analytic subspaces in fractal $L^2$-spaces, J. Anal. Math., 75 (1998), 185-228. doi: 10.1007/BF02788699.
[56]	P. E. T. Jorgensen and M.-S. Song, Markov chains and generalized wavelet multiresolutions, J. Anal., 26 (2018), 259-283. doi: 10.1007/s41478-018-0139-9.
[57]	P. E. T. Jorgensen and F. Tian, Random measures in infinite-dimensional dynamics, in Advanced Topics in Mathematical Analysis, CRC Press, Boca Raton, FL, (2019), 1–38.
[58]	P. E. Jorgensen and F. Tian, Duality for Gaussian Processes from Random Signed Measures, chapter 2, 23–56, John Wiley & Sons, Ltd, 2018
[59]	O. G. Jø rsboe, Equivalence or Singularity of Gaussian Measures on Function Spaces, Various Publications Series, No. 4, Matematisk Institut, Aarhus Universitet, Aarhus, 1968.
[60]	M. T. Jury and R. T. W. Martin, Extremal multipliers of the Drury-Arveson space, Proc. Amer. Math. Soc., 146 (2018), 4293-4306. doi: 10.1090/proc/14062.
[61]	M. Katori and T. Shirai, Partial isometries, duality, and determinantal point processes, arXiv e-prints, arXiv: 1903.04945.
[62]	A. N. Kolmogorov, On logical foundations of probability theory, in Probability Theory and Mathematical Statistics (Tbilisi, 1982), vol. 1021 of Lecture Notes in Math., Springer, Berlin, (1983), 1–5. doi: 10.1007/BFb0072897.
[63]	A. Kulesza and B. Taskar, Determinantal Point Processes for Machine Learning, Foundations and Trends in Machine Learning, Now Publishers, 2012, https://books.google.com/books?id=FY-6lAEACAAJ. doi: 10.1561/9781601986290.
[64]	Z. Li, Y. Fan and L. Ying, Multilevel fine-tuning: Closing generalization gaps in approximation of solution maps under a limited budget for training data, Multiscale Model. Simul., 19 (2021), 344-373. doi: 10.1137/20M1326404.
[65]	M. A. Lifshits, Gaussian Random Functions, vol. 322 of Mathematics and its Applications, Kluwer Academic Publishers, Dordrecht, 1995. doi: 10.1007/978-94-015-8474-6.
[66]	W. L. Liu, S. L. Lü and F. B. Liang, Kernel density discriminant method based on geodesic distance, J. Fuzhou Univ. Nat. Sci. Ed., 39 (2011), 807–810,818.
[67]	K. R. Parthasarathy and K. Schmidt, Stable positive definite functions, Trans. Amer. Math. Soc., 203 (1975), 161-174. doi: 10.1090/S0002-9947-1975-0370681-X.
[68]	V. I. Paulsen and M. Raghupathi, An Introduction to the Theory of Reproducing Kernel Hilbert Spaces, vol. 152* of Cambridge Studies in Advanced Mathematics*, Cambridge University Press, Cambridge, 2016. doi: 10.1017/CBO9781316219232.
[69]	R. Rochberg, Complex hyperbolic geometry and Hilbert spaces with complete Pick kernels, J. Funct. Anal., 276 (2019), 1622-1679. doi: 10.1016/j.jfa.2018.08.017.
[70]	W. Rudin, Function Theory in the Unit Ball of ${\mathbb{C}} ^n$, Classics in Mathematics, Springer-Verlag, Berlin, 2008, Reprint of the 1980 edition.
[71]	A. A. Sabree, Positive Definite Kernels, Harmonic Analysis, and Boundary Spaces: Drury-Arveson Theory, and Related, PhD thesis, The University of Iowa, 2019.
[72]	S. Saitoh, Integral Transforms, Reproducing Kernels and their Applications, vol. 369 of Pitman Research Notes in Mathematics Series, Longman, Harlow, 1997.
[73]	S. Saitoh, A reproducing kernel theory with some general applications, in Mathematical Analysis, Probability and Applications–-Plenary Lectures, vol.177 of Springer Proc. Math. Stat., Springer, [Cham], (2016), 151–182. doi: 10.1007/978-3-319-41945-9_6.
[74]	S. Smale and D.-X. Zhou, Shannon sampling. II. Connections to learning theory, Appl. Comput. Harmon. Anal., 19 (2005), 285-302. doi: 10.1016/j.acha.2005.03.001.
[75]	S. Smale and D.-X. Zhou, Learning theory estimates via integral operators and their approximations, Constr. Approx., 26 (2007), 153-172. doi: 10.1007/s00365-006-0659-y.
[76]	G. Steidl, Supervised learning by support vector machines, in Handbook of Mathematical Methods in Imaging., Vol. 1, 2, 3, Springer, New York, (2015), 1393–1453.
[77]	Y. Wang and J. Xiao, Well/ill-posedness for the dissipative Navier-Stokes system in generalized Carleson measure spaces, Adv. Nonlinear Anal., 8 (2019), 203-224. doi: 10.1515/anona-2016-0042.
[78]	C. Wickman and K. Okoudjou, Duality and geodesics for probabilistic frames, Linear Algebra Appl., 532 (2017), 198-221. doi: 10.1016/j.laa.2017.05.034.
[79]	L. Yang, Carleson type measures supported on $(-1, 1)$ and Hankel matrices, J. Math. Res. Appl., 38 (2018), 471-477.
[80]	Y. Ying, Y. Lian, S. Tang and W. K. Liu, Enriched reproducing kernel particle method for fractional advection-diffusion equation, Acta Mech. Sin., 34 (2018), 515-527. doi: 10.1007/s10409-017-0742-z.