The space of probability densities is an infinite-dimensional Riemannian manifold, with Riemannian metrics in two flavors: Wasserstein and Fisher-Rao. The former is pivotal in optimal mass transport (OMT), whereas the latter occurs in information geometry--the differential geometric approach to statistics. The Riemannian structures restrict to the submanifold of multivariate Gaussian distributions, where they induce Riemannian metrics on the space of covariance matrices.
Here we give a systematic description of classical matrix decompositions (or factorizations) in terms of Riemannian geometry and compatible principal bundle structures. Both Wasserstein and Fisher-Rao geometries are discussed. The link to matrices is obtained by considering OMT and information geometry in the category of linear transformations and multivariate Gaussian distributions. This way, OMT is directly related to the polar decomposition of matrices, whereas information geometry is directly related to the $QR$, Cholesky, spectral, and singular value decompositions. We also give a coherent description of gradient flow equations for the various decompositions; most flows are illustrated in numerical examples.
The paper is a combination of previously known and original results. As a survey it covers the Riemannian geometry of OMT and polar decompositions (smooth and linear category), entropy gradient flows, and the Fisher-Rao metric and its geodesics on the statistical manifold of multivariate Gaussian distributions. The original contributions include new gradient flows associated with various matrix decompositions, new geometric interpretations of previously studied isospectral flows, and a new proof of the polar decomposition of matrices based an entropy gradient flow.
Citation: |
Figure 1. Illustration of the geometry of the polar decomposition of diffeomorphisms. The element $\nabla\phi$ in the factorization $\varphi=\nabla\phi\circ\psi$ is obtained at the intersection of the polar cone and the fiber of $\mu_1=\pi(\varphi)$. To compute $\nabla\phi$, one may start at $\varphi$ and follow a gradient flow constrained to the fiber of $\mu_1$ (vertical gradient flow, see $\S 2.2.1$), or one may take a gradient flow of a functional on the space of densities that approaches $\mu_1$ (entropy gradient flow, see $\S 2.2.2$) and lift it to a corresponding gradient flow on the polar cone (lifted gradient flow, see $\S 2.2.3$).
M. Adler
, On a trace functional for formal pseudo-differential operators and the symplectic structure of the Korteweg-deVries type equations, Invent. Math., 50 (1978)
, 219-248.
doi: 10.1007/BF01410079.![]() ![]() ![]() |
|
F. Alvarez
, J. Bolte
and O. Brahic
, Hessian Riemannian gradient flows in convex programming, SIAM J. Control and Optim., 43 (2004)
, 477-501.
doi: 10.1137/S0363012902419977.![]() ![]() ![]() |
|
S. Amari and H. Nagaoka, Methods of Information Geometry, Amer. Math. Soc., Providence, RI, 2000.
![]() ![]() |
|
S. Angenent
, S. Haker
and A. Tannenbaum
, Minimizing flows for the Monge-Kantorovich problem, SIAM J. Math. Anal., 35 (2003)
, 61-97.
doi: 10.1137/S0036141002410927.![]() ![]() ![]() |
|
C. Atkinson
and A. Mitchell
, Rao's distance measure, Indian J. Stat. A, 43 (1981)
, 345-365.
![]() ![]() |
|
F. Barbaresco, Information geometry of covariance matrix: Cartan-siegel homogeneous bounded domains, mostow/berger fibration and frechet median, in Matrix Information Geometry, Springer, 2013,199-255.
doi: 10.1007/978-3-642-30232-9_9.![]() ![]() ![]() |
|
R. H. Bartels
and G. Stewart
, Solution of the matrix equation $AX+XB=C$, Comm. ACM, 15 (1972)
, 820-826.
doi: 10.1145/361573.361582.![]() ![]() |
|
J.-D. Benamou
and Y. Brenier
, A computational fluid mechanics solution to the monge-kantorovich mass transfer problem, Numer. Math., 84 (2000)
, 375-393.
doi: 10.1007/s002110050002.![]() ![]() ![]() |
|
J. -D. Benamou, Y. Brenier and A. Oberman, Advances in Numerical Optimal Transportation, Technical Report 15w5067, Banff International Research Station, 2015.
![]() |
|
A. Bloch
, Estimation, principal components and Hamiltonian systems, Sys. & Cont. Lett., 6 (1985)
, 103-108.
doi: 10.1016/0167-6911(85)90005-2.![]() ![]() ![]() |
|
A. M. Bloch
, Steepest descent, linear programming and Hamiltonian flows, Contemp. Math. AMS, 114 (1990)
, 77-88.
doi: 10.1090/conm/114/1097866.![]() ![]() ![]() |
|
A. M. Bloch
, R. W. Brockett
and T. S. Ratiu
, Completely integrable gradient flows, Comm. Math. Phys., 147 (1992)
, 57-74.
doi: 10.1007/BF02099528.![]() ![]() ![]() |
|
Y. Brenier
, Polar factorization and monotone rearrangement of vector-valued functions, Comm. Pure Appl. Math., 44 (1991)
, 375-417.
doi: 10.1002/cpa.3160440402.![]() ![]() ![]() |
|
R. W. Brockett
, Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems, Linear Algebra Appl., 146 (1991)
, 79-91.
doi: 10.1016/0024-3795(91)90021-N.![]() ![]() ![]() |
|
J. Burbea, Informative Geometry of Probability Spaces, Technical report, DTIC Document, 1984.
![]() |
|
J. Burbea
and C. R. Rao
, Entropy differential metric, distance and divergence measures in probability spaces: A unified approach, J. Multivariate Anal., 12 (1982)
, 575-596.
doi: 10.1016/0047-259X(82)90065-3.![]() ![]() ![]() |
|
J. C. Butcher, Numerical Methods for Ordinary Differential Equations, 2nd edition, John Wiley & Sons Ltd., Chichester, 2008.
doi: 10.1002/9780470753767.![]() ![]() ![]() |
|
L. A. Caffarelli
, The regularity of mappings with a convex potential, J. Amer. Math. Soc., 5 (1992)
, 99-104.
doi: 10.1090/S0894-0347-1992-1124980-8.![]() ![]() ![]() |
|
M. P. Calvo
, A. Iserles
and A. Zanna
, Numerical solution of isospectral flows, Math. Comp., 66 (1997)
, 1461-1486.
doi: 10.1090/S0025-5718-97-00902-2.![]() ![]() ![]() |
|
M. Calvo
and J. M. Oller
, A distance between multivariate normal distributions based in an embedding into the siegel group, J. Multivariate Anal., 35 (1990)
, 223-242.
doi: 10.1016/0047-259X(90)90026-E.![]() ![]() ![]() |
|
E. Celledoni
, H. Marthinsen
and B. Owren
, An introduction to Lie group integrators-basics, new developments and applications, J. Comput. Phys., 257 (2014)
, 1040-1061.
doi: 10.1016/j.jcp.2012.12.031.![]() ![]() ![]() |
|
N. N. Čencov, Statistical Decision Rules and Optimal Inference, Amer. Math. Soc., Providence, R. I., 1982.
![]() ![]() |
|
M. T. Chu
, The generalized Toda flow, the QR algorithm and the center manifold theory, SIAM J. Alg. Discrete Meth., 5 (1984)
, 187-201.
doi: 10.1137/0605020.![]() ![]() ![]() |
|
M. T. Chu
, Matrix differential equations: A continuous realization process for linear algebra problems, Nonlin. Anal.: Theor. Meth. & Appl., 18 (1992)
, 1125-1146.
doi: 10.1016/0362-546X(92)90157-A.![]() ![]() ![]() |
|
M. T. Chu
, A list of matrix flows with applications, Fields Institute Communications, 3 (1994)
, 87-97.
![]() ![]() |
|
M. T. Chu
, Scaled Toda-like flows, Linear Algebra Appl., 215 (1995)
, 261-273.
doi: 10.1016/0024-3795(93)00091-D.![]() ![]() ![]() |
|
M. T. Chu
, Linear algebra algorithms as dynamical systems, Acta Numer., 17 (2008)
, 1-86.
doi: 10.1017/S0962492906340019.![]() ![]() ![]() |
|
M. T. Chu
and K. R. Driessel
, Constructing symmetric nonnegative matrices with prescribed eigenvalues by differential equations, SIAM J. Math. Anal., 22 (1991)
, 1372-1387.
doi: 10.1137/0522088.![]() ![]() ![]() |
|
M. T. Chu
and L. K. Norris
, Isospectral flows and abstract matrix factorizations, SIAM J. Numer. Anal., 25 (1988)
, 1383-1391.
doi: 10.1137/0725080.![]() ![]() ![]() |
|
B. Clarke
, The metric geometry of the manifold of Riemannian metrics over a closed manifold, Calc. Var. PDE, 39 (2010)
, 533-545.
doi: 10.1007/s00526-010-0323-5.![]() ![]() ![]() |
|
B. Clarke
, The completion of the manifold of Riemannian metrics, J. Differential Equations, 93 (2013)
, 203-268.
doi: 10.4310/jdg/1361800866.![]() ![]() ![]() |
|
P. Deift
, J. Demmel
, L.-C. Li
and C. Tomei
, The bidiagonal singular value decomposition and Hamiltonian mechanics, SIAM J. Numer. Anal., 28 (1991)
, 1463-1516.
doi: 10.1137/0728076.![]() ![]() ![]() |
|
P. Deift
, T. Nanda
and C. Tomei
, Ordinary differential equations and the symmetric eigenvalue problem, SIAM J. Numer. Anal., 20 (1983)
, 1-22.
doi: 10.1137/0720001.![]() ![]() ![]() |
|
P. Deift
, L. Li
, T. Nanda
and C. Tomei
, The Toda flow on a generic orbit is integrable, Comm. Pure Appl. Math., 39 (1986)
, 183-232.
doi: 10.1002/cpa.3160390203.![]() ![]() ![]() |
|
D. G. Ebin
, On the space of Riemannian metrics, Bull. Amer. Math. Soc., 74 (1968)
, 1001-1003.
doi: 10.1090/S0002-9904-1968-12115-9.![]() ![]() |
|
D. G. Ebin
and J. E. Marsden
, Groups of diffeomorphisms and the notion of an incompressible fluid, Ann. of Math., 92 (1970)
, 102-163.
doi: 10.2307/1970699.![]() ![]() ![]() |
|
R. A. Fisher
, On the mathematical foundations of theoretical statistics, Breakthroughs in Statistics: Part of the series Springer Series in Statistics, (1992)
, 11-44.
doi: 10.1007/978-1-4612-0919-5_2.![]() ![]() |
|
H. Flaschka
, The Toda lattice. Ⅱ. existence of integrals, Physical Review B, 9 (1974)
, 1924-1925.
doi: 10.1103/PhysRevB.9.1924.![]() ![]() ![]() |
|
D. S. Freed
and D. Groisser
, The basic geometry of the manifold of Riemannian metrics and of its quotient by the diffeomorphism group, Michigan Math. J., 36 (1989)
, 323-344.
doi: 10.1307/mmj/1029004004.![]() ![]() ![]() |
|
T. Friedrich
, Die Fisher-information und symplektische strukturen, Math. Nachr., 153 (1991)
, 273-296.
doi: 10.1002/mana.19911530125.![]() ![]() ![]() |
|
N. H. Getz
and J. E. Marsden
, Dynamical methods for polar decomposition and inversion of matrices, Linear Algebra Appl., 258 (1997)
, 311-343.
doi: 10.1016/S0024-3795(96)00235-2.![]() ![]() ![]() |
|
O. Gil-Medrano
and P. W. Michor
, The Riemannian manifold of all Riemannian metrics, Quart. J. of Math., 42 (1991)
, 183-202.
doi: 10.1093/qmath/42.1.183.![]() ![]() ![]() |
|
G. H. Golub
and H. A. van der Vorst
, Eigenvalue computation in the 20th century, J. Comput. Appl. Math., 123 (2000)
, 35-65.
doi: 10.1016/S0377-0427(00)00413-1.![]() ![]() ![]() |
|
R. E. Greene
and K. Shiohama
, Diffeomorphisms and volume-preserving embeddings of noncompact manifolds, Trans. Amer. Math. Soc., 255 (1979)
, 403-414.
doi: 10.1090/S0002-9947-1979-0542888-3.![]() ![]() ![]() |
|
R. S. Hamilton
, The inverse function theorem of Nash and Moser, Bull. Amer. Math. Soc. (N.S.), 7 (1982)
, 65-222.
doi: 10.1090/S0273-0979-1982-15004-2.![]() ![]() ![]() |
|
U. Helmke
and J. Moore
, Singular-value decomposition via gradient and self-equivalent flows, Linear Algebra Appl., 169 (1992)
, 223-248.
doi: 10.1016/0024-3795(92)90180-I.![]() ![]() ![]() |
|
U. Helmke
, J. Moore
and J. Perkins
, Dynamical systems that compute balanced realizations and the singular value decomposition, SIAM J. Matrix Anal. Appl., 15 (1994)
, 733-754.
doi: 10.1137/S0895479891222490.![]() ![]() ![]() |
|
R. Hermann
, A sufficient condition that a mapping of Riemannian manifolds be a fibre bundle, Proc. Amer. Math. Soc., 11 (1960)
, 236-242.
doi: 10.1090/S0002-9939-1960-0112151-4.![]() ![]() ![]() |
|
J. H. Hodges
, Some matrix equations over a finite field, Annali di Matematica Pura ed Applicata, 44 (1957)
, 245-250.
doi: 10.1007/BF02415202.![]() ![]() ![]() |
|
R. Jordan
, D. Kinderlehrer
and F. Otto
, The variational formulation of the Fokker-Planck equation, SIAM J. Math. Anal., 29 (1998)
, 1-17.
doi: 10.1137/S0036141096303359.![]() ![]() ![]() |
|
B. Khesin
, J. Lenells
, G. Misiolek
and S. C. Preston
, Geometry of diffeomorphism groups, complete integrability and geometric statistics, Geom. Funct. Anal., 23 (2013)
, 334-366.
doi: 10.1007/s00039-013-0210-2.![]() ![]() ![]() |
|
B. Khesin and R. Wendt, The Geometry of Infinite-dimensional Groups, vol. 51, Springer-Verlag, Berlin, 2009.
![]() ![]() |
|
W. Kratz
and M. Tentler
, Recursion formulae for the characteristic polynomial of symmetric banded matrices, Linear Algebra Appl., 428 (2008)
, 2482-2500.
doi: 10.1016/j.laa.2007.11.024.![]() ![]() ![]() |
|
A. Kriegl and P. W. Michor, The Convenient Setting of Global Analysis, vol. 53, American Mathematical Society, Providence, RI, 1997.
doi: 10.1090/surv/053.![]() ![]() ![]() |
|
S. Lang, Fundamentals of Differential Geometry, Springer-Verlag, New York, 1999.
doi: 10.1007/978-1-4612-0541-8.![]() ![]() ![]() |
|
J. Lott
, Some geometric calculations on Wasserstein space, Comm. Math. Phys., 277 (2008)
, 423-437.
doi: 10.1007/s00220-007-0367-3.![]() ![]() ![]() |
|
J. E. Marsden and T. S. Ratiu, Introduction to Mechanics and Symmetry, vol. 17, Springer-Verlag, New York, 1999.
doi: 10.1007/978-0-387-21792-5.![]() ![]() ![]() |
|
R. J. McCann
, Polar factorization of maps on Riemannian manifolds, Geom. Funct. Anal., 11 (2001)
, 589-608.
doi: 10.1007/PL00001679.![]() ![]() ![]() |
|
P. W. Michor, Topics in Differential Geometry, vol. 93, American Mathematical Society, Providence, RI, 2008.
doi: 10.1090/gsm/093.![]() ![]() ![]() |
|
K. Modin
, Generalized Hunter-Saxton equations, optimal information transport, and factorization of diffeomorphisms, J. Geom. Anal., 25 (2015)
, 1306-1334.
doi: 10.1007/s12220-014-9469-2.![]() ![]() ![]() |
|
G. Monge, Mémoire sur la théorie des déblais et de remblais, 1781.
![]() |
|
J. Moser
, On the volume elements on a manifold, Trans. Amer. Math. Soc., 120 (1965)
, 286-294.
doi: 10.1090/S0002-9947-1965-0182927-5.![]() ![]() ![]() |
|
J. Moser, Finitely many mass points on the line under the influence of an exponential potential-an integrable system, in Dynamical systems, theory and applications, Springer, 1975,467-497.
![]() ![]() |
|
J. Moser
and A. P. Veselov
, Discrete versions of some classical integrable systems and factorization of matrix polynomials, Comm. Math. Phys., 139 (1991)
, 217-243.
doi: 10.1007/BF02352494.![]() ![]() ![]() |
|
F. Otto
, The geometry of dissipative evolution equations: The porous medium equation, Comm. Partial Differential Equations, 26 (2001)
, 101-174.
doi: 10.1081/PDE-100002243.![]() ![]() ![]() |
|
B. N. Parlett,
The Symmetric Eigenvalue Problem, SIAM, 1980.
![]() ![]() |
|
P. Petersen, Riemannian Geometry, vol. 171 of Graduate Texts in Mathematics, 2nd edition, Springer, New York, 2006.
![]() ![]() |
|
G. Peyré
, Entropic approximation of Wasserstein gradient flows, SIAM J. on Imag. Sci., 8 (2015)
, 2323-2351.
doi: 10.1137/15M1010087.![]() ![]() ![]() |
|
C. Rao
, Information and the accuracy attainable in the estimation of statistical parameters, Bull. Calcutta Math. Soc., 37 (1945)
, 81-91.
![]() ![]() |
|
C. R. Rao
, Differential metrics in probability spaces, Diff. Geom. Stat. Inference, 10 (1987)
, 217-240.
![]() |
|
A. Reyman
and M. Semenov-Tian-Shansky
, Reduction of Hamiltonian systems, affine Lie algebras and Lax equations, Inv. Math., 54 (1979)
, 81-100.
doi: 10.1007/BF01391179.![]() ![]() ![]() |
|
H. Rutishauser
, Ein infinitesimales analogon zum quotienten-differenzen-algorithmus, Archiv der Mathematik, 5 (1954)
, 132-137.
doi: 10.1007/BF01899329.![]() ![]() ![]() |
|
H. Rutishauser
, Solution of eigenvalue problems with the LR-transformation, Nat. Bur. Standards Appl. Math. Ser, 1958 (1958)
, 47-81.
![]() ![]() |
|
H. Shima,
The Geometry of Hessian Structures, World Scientific Publishing Co. Inc., Hackensack, NJ, 2007.
doi: 10.1142/9789812707536.![]() ![]() ![]() |
|
L. T. Skovgaard
, A Riemannian geometry of the multivariate normal model, Scandinavian J. of Stat., 11 (1984)
, 211-223.
![]() ![]() |
|
J. Sylvester
, Sur l'equations en matrices $px=xq$, C.R. Acad. Sci., 99 (1884)
, 115-116.
![]() |
|
W. Symes
, Hamiltonian group actions and integrable systems, Phys. D, 1 (1980)
, 339-374.
doi: 10.1016/0167-2789(80)90017-2.![]() ![]() ![]() |
|
W. Symes
, The QR algorithm and scattering for the finite nonperiodic Toda lattice, Phys. D, 4 (1982)
, 275-280.
doi: 10.1016/0167-2789(82)90069-0.![]() ![]() ![]() |
|
A. Takatsu
, Wasserstein geometry of Gaussian measures, Osaka J. Math., 48 (2011)
, 1005-1026.
![]() ![]() |
|
M. Toda
, Waves in nonlinear lattice, Selected Papers of Morikazu Toda, 18 (1993)
, 112-138.
doi: 10.1142/9789814354332_0017.![]() ![]() |
|
C. Tomei
, The Toda lattice, old and new, J. Geom. Mech., 5 (2013)
, 511-530.
doi: 10.3934/jgm.2013.5.511.![]() ![]() ![]() |
|
C. Villani, Optimal Transport: Old and New, vol. 338, Springer-Verlag, Berlin, 2009.
doi: 10.1007/978-3-540-71050-9.![]() ![]() ![]() |
|
D. S. Watkins
, Isospectral flows, SIAM Rev., 26 (1984)
, 379-391.
doi: 10.1137/1026075.![]() ![]() ![]() |
|
D. S. Watkins
, Some perspectives on the eigenvalue problem, SIAM Rev., 35 (1993)
, 430-471.
doi: 10.1137/1035090.![]() ![]() ![]() |
Illustration of the geometry of the polar decomposition of diffeomorphisms. The element
Evolution of the matrix elements of
Convergence towards the limit
Evolution of the lifted gradient flow in Example 2. Notice that
Convergence towards the limit
Evolution of the lifted gradient flow in Example 3. Notice that
Convergence towards the limit
Phase diagram of equation (86) for geodesics on
Evolution of the horizontal gradient flow in Example 4.
Convergence of