Sub-Riemannian landmark matching and its interpretation as residual neural networks

Erik Jansson; Klas Modin

doi:10.3934/jcd.2025004

Article Contents

2025, Volume 12, Issue 3: 467-490. Doi: 10.3934/jcd.2025004

This issue Previous Article Computer assisted existence proofs for parameterized cycle-to-cycle connections Next Article

Sub-Riemannian landmark matching and its interpretation as residual neural networks

Erik Jansson^, and
Klas Modin

Department of Mathematical Sciences, Chalmers University of Technology & University of Gothenburg, Sweden

^*Corresponding author: Erik Jansson
^*Corresponding author: Erik Jansson

Received: September 2024

Revised: February 2025

Early access: April 27, 2025

Published online: April 27, 2025

Abstract / Introduction Full Text(HTML) Figure(13) / Table(2) Related Papers Cited by

Abstract

The problem of finding a time-dependent vector field which warps an initial set of points to a target set is common in shape analysis. It is an example of a problem in the diffeomorphic shape matching regime, and can be thought of as a spatial discretization of diffeomorphic image matching. In this paper, we consider landmark matching modified by restricting the set of available vector fields in the sense that vector fields are parametrized by a set of controls. We determine the geometric setting of the problem, referred to as sub-Riemannian landmark matching, and derive the equations of motion for the controls. We provide two computational algorithms and demonstrate them in numerical examples. In particular, the experiments highlight the importance of the regularization term. A strong motivation is that sub-Riemannian landmark matching have connections with neural networks, in particular the interpretation of residual neural networks as time discretizations of continuous control problems. It allows shape analysis practitioners to think about neural networks in terms of diffeomorphic landmark matching, thereby providing a bridge between the two fields.

Keywords:

Mathematics Subject Classification: Primary: 68T07, 53C17, 70H99, 70G45.

Citation:

Full Text(HTML)

Figure 1. Illustration of the $ \mathcal E(\mathcal S) $ sub-bundle of $ T\operatorname{Diff}(M) $ in two different cases

Download: Full-size image PowerPoint slide

Figure 2. The left fish is the initial image, and the right one is the target. Note the structural similarities between the fish, implying that it should be possible to warp one to the other

Download: Full-size image PowerPoint slide

Figure 3. The images in Figure 2 approximated by landmarks on the flat torus

Download: Full-size image PowerPoint slide

Figure 4. The result of applying the shooting method with constant vector fields. We illustrate both how the initial landmarks are transported and the resulting grid deformation. The deformed landmarks are only a translation of the original landmarks

Download: Full-size image PowerPoint slide

Figure 5. The result of applying the shooting method with four vector fields that form an algebra

Download: Full-size image PowerPoint slide

Figure 6. The result of applying the shooting method with four vector fields that do not form an algebra. The resulting deformation is a drastic improvement to the previous examples

Download: Full-size image PowerPoint slide

Figure 7. The effect of changing the integration step size $ h $. The blue hexagons are the initial landmarks, the orange circles are the targets and the green squares are the deformed landmarks. Note that the matching improves as $ h $ decreases. This is because the vector fields do not form an algebra

Download: Full-size image PowerPoint slide

Figure 8. The result of applying the shooting method with more complicated vector fields, taken to build truncated Fourier expansions

Download: Full-size image PowerPoint slide

Figure 9. Two artificial datasets. The orange dots are classified as 1, the blue dots as 0

Download: Full-size image PowerPoint slide

Figure 10. The result of applying the shooting method to the spiral data set using a lower regularization parameter. The classification boundary consists of the black horizontal lines: points between them are classified as 1, and points outside them are classified as 0

Download: Full-size image PowerPoint slide

Figure 11. The result of applying the shooting method to the spiral data set using a higher regularization parameter. The classification boundary consists of the black horizontal lines: points between them are classified as 1, and points outside them are classified as 0

Download: Full-size image PowerPoint slide

Figure 12. Schematic illustration of layers in two different neural network architectures

Download: Full-size image PowerPoint slide

Figure 13. A layer in the neural network control-affine case. Note that the layout is dependent on the choice of temporal discretization. In this case, forward Euler is used, resulting in a ResNet-like structure

Download: Full-size image PowerPoint slide

Table 1. The impact of regularization strength on classification performance

$ \sigma $ (times $ 1/(4\pi^2) $)	Correct %	Correct %, new data
$ 0.2 $	$ 86.5 $	$ 83.2 $
$ 1.0 $	$ 87.8 $	$ 86.9 $
$ 10.0 $	$ 83.7 $	$ 82.8 $
$ 100.0 $	$ 78.8 $	$ 75.4 $
$ 1000.0 $	$ 60.2 $	$ 53.5 $

| Show Table

DownLoad: CSV

Table 2. Dictionary between sub-Riemannian landmark matching and ResNets

Deep Learning	Landmark matching
Images	Landmarks
Meta-images	Images
Training network	Shooting method
Testing	Warping new landmarks
Classification layer	Forward model
Weights and biases	Control parameters in $ \mathcal{U} $
Neural architecture	ODE Discretization

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	A. A. Agrachev and Y. L. Sachkov, Control Theory from the Geometric Viewpoint, Encyclopaedia Math. Sci., 87. Control Theory Optim., Ⅱ. Springer-Verlag, Berlin, 2004.
[2]	A. A. Agrachev and A. Sarychev, Control on the manifolds of mappings with a view to the deep learning, J. Dyn. Control Syst., 28 (2022), 989-1008. doi: 10.1007/s10883-021-09561-2.
[3]	S. Arguillère, E. Trélat, A. Trouvé and L. Younes, Shape deformation analysis from the optimal control viewpoint, J. Math. Pures. Appl., 104 (2015), 139-178. doi: 10.1016/j.matpur.2015.02.004.
[4]	S. Arguillère, E. Trélat, A. Trouvé and L. Younes, Registration of multiple shapes using constrained optimal control, SIAM J. Imaging Sci., 9 (2016), 344-385. doi: 10.1137/15M1006726.
[5]	K. Beauchard, J. Le Borgne and F. Marbach, On expansions for nonlinear systems error estimates and convergence issues, C. R. Math., 361 (2023), 97-189. doi: 10.5802/crmath.395.
[6]	M. F. Beg, M. I. Miller, A. Trouvé and L. Younes, Computing large deformation metric mappings via geodesic flows of diffeomorphisms, Int. J. Comput. Vis., 61 (2005), 139-157. doi: 10.1023/B:VISI.0000043755.93987.aa.
[7]	B. Ben Amor, S. Arguillere and L. Shao, ResNet-LDDMM: Advancing the LDDMMframework using deep residual networks, IEEE Trans. Pattern Anal. Mach. Intell., 45 (2023), 3707-3720. doi: 10.1109/TPAMI.2022.3174908.
[8]	C. M. Bishop, Pattern Recognition and Machine Learning, Inf. Sci. Stat., Springer, New York, 2006.
[9]	A. Bistoquet, J. Oshinski and O. Skrinjar, Myocardial deformation recovery from cine MRI using a nearly incompressible biventricular model, Med. Image Anal., 12 (2008), 69-85. doi: 10.1016/j.media.2007.10.009.
[10]	A. Bloch, P. S. Krishnaprasad, J. E. Marsden and T. S. Ratiu, The Euler–Poincaré equations and double bracket dissipation, Commun. Math. Phys., 175 (1996), 1-42. doi: 10.1007/BF02101622.
[11]	M. Bruveris, Geometry of Diffeomorphism Groups and Shape Matching, Ph.D. thesis, Imperial College London, 2012.
[12]	M. Bruveris and D. D. Holm, Geometry of image registration: The diffeomorphism group and momentum maps, Geometry, Mechanics, and Dynamics, Fields Inst. Commun., Springer, New York, 73 (2015), 19-56. doi: 10.1007/978-1-4939-2441-7_2.
[13]	E. Celledoni, M. J. Ehrhardt, R. I. Etmann, Ca nd McLachlan, B. Owren, C.-B. Schönlieb and F. Sherry, Structure-preserving deep learning, Eur. J. Appl. Math., 32 (2021), 888-936. doi: 10.1017/S0956792521000139.
[14]	E. Celledoni, H. Glöckner, J. N. Riseth and A. Schmeding, Deep neural networks on diffeomorphism groups for optimal shape reparametrization, BIT Numer. Math., 63 (2023), Paper No. 50, 38 pp. doi: 10.1007/s10543-023-00989-5.
[15]	C. Ceritoglu, X. Tang, M. Chow, D. Hadjiabadi, D. Shah, T. Brown, M. Burhanullah, H. Trinh, J. Hsu, K. Ament, D. Crocetti, S. Mori, S. Mostofsky, S. Yantis, M. I. Miller and T. J. Ratnanather, Computational analysis of LDDMM for brain mapping, Front. Neurosci., 7 (2013), 151. doi: 10.3389/fnins.2013.00151.
[16]	A. Cifor, L. Risser, D. Chung, E. M. Anderson and J. A. Schnabel, Hybrid feature-based diffeomorphic registration for tumor tracking in 2-D liver ultrasound images, IEEE Trans. Med. Imaging, 32 (2013), 1647-1656. doi: 10.1109/TMI.2013.2262055.
[17]	N. Ganaba, Deep learning: Hydrodynamics, and Lie-Poisson Hamilton-Jacobi theory, arXiv: 2105.09542v2, (2021).
[18]	J. A. Glaunes, Transport par Difféomorphismes de Points, Demesures et de Courants Pour la Comparaisonde Formes et L'Anatomie Numérique, Ph.D. thesis, Université Paris 13, 2005.
[19]	U. Grenander, General Pattern Theory, Clarendon Press, Oxford, 1993.
[20]	B. Gris, S. Durrleman and A. Trouvé, A Sub-Riemannian modular framework for diffeomorphism-based analysis of shape ensembles, SIAM J. Imaging Sci., 11 (2018), 802-833. doi: 10.1137/16M1076733.
[21]	R. S. Hamilton, The inverse function theorem of Nash and Moser, Bull. Am. Math. Soc., 7 (1982), 65-222. doi: 10.1090/S0273-0979-1982-15004-2.
[22]	K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 770-778.
[23]	L. Hörmander, Hypoelliptic second order differential equations, Acta Math., 119 (1967), 147-171. doi: 10.1007/BF02392081.
[24]	T. Hytönen, J. van Neerven, M. Veraar and L. Weis, Analysis in Banach Spaces: Volume Ⅰ: Martingales and Littlewood-Paley Theory, Springer, 2016.
[25]	S. C. Joshi and M. I. Miller, Landmark matching via large deformation diffeomorphisms, IEEE Trans. Image Process., 9 (2000), 1357-1370. doi: 10.1109/83.855431.
[26]	V. Jurdjevic, Optimal control, geometry, and mechanics, Mathematical Control Theory, (1999), 227-267. doi: 10.1007/978-1-4612-1416-8_7.
[27]	Y. LeCun, A theoretical framework for back-propagation, D. Touretzky, G. Hinton and T. Sejnowski, Proceedings of the 1988 connectionist models summer school, Morgan Kaufmann, CMU, Pittsburgh, Pa, (1988), 21-28.
[28]	Q. Li, L. Chen, T. Cheng and E. Weinan, Maximum principle based algorithms for deep learning, J. Mach. Learn. Res., 18 (2017), 5998-6026.
[29]	T. Mansi, X. Pennec, M. Sermesant, H. Delingette and N. Ayache, iLogDemons: A demons-based registration algorithm for tracking incompressible elastic biological tissues, Int. J. Comput. Vis., 92 (2010), 92-111. doi: 10.1007/s11263-010-0405-z.
[30]	J. E. Marsden and T. Ratiu, Introduction to Mechanics and Symmetry, Springer, 1999.
[31]	W. Miller, Computational complexity and numerical stability, Proceedings of the Sixth Annual ACM Symposium on Theory of Computing, Association for Computing Machinery, (1974), 317–322.
[32]	P. K. Mogensen and A. N. Riseth, Optim: A mathematical optimization package for Julia, J. Open Source Softw., 3 (2018), 615. doi: 10.21105/joss.00615.
[33]	J. Nielsen Clelland, C. Moseley and G. R. Wilkens, Geometry of control-affine systems, Symmetry Integr. Geom., 5 (2009), Paper 095, 28 pp.
[34]	H. Owhadi, Do ideas have shape? Idea registration as the continuous limit of artificial neural networks, Physica D, 444 (2023), 133592, 41 pp. doi: 10.1016/j.physd.2022.133592.
[35]	O. Öktem, C. Chong, D. Nevzat, R. Pradeep and B. Chandrajit, Shape based image reconstruction using linearized deformations, Inverse Probl., 33 (2017), 035004, 33 pp. doi: 10.1088/1361-6420/aa55af.
[36]	A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai and S. Chintala, PyTorch: An imperative style, high-performance deep learning library, Advances in Neural Information Processing Systems 32, (2019), 8024-8035.
[37]	C. Rackauckas and Q. Nie, Differentialequations.jl–A performant and feature-rich ecosystem for solving differential equations in Julia, J. Open Res. Softw., 5 (2017). doi: 10.5334/jors.151.
[38]	J. Revels, M. Lubin and T. Papamarkou, Forward-mode automatic differentiation in Julia, CoRR, (2016).
[39]	L. Risser, F. X. Vialard, H. Y. Baluwala and J. A. Schnabel, Piecewise-diffeomorphic image registration: Application to the motion estimation between 3D CT lung images with sliding conditions, Med. Image Anal., 17 (2013), 182-193. doi: 10.1016/j.media.2012.10.001.
[40]	A. Scagliotti, Deep learning approximation of diffeomorphisms via linear-control systems, Math. Control. Relat. Fields, 13 (2023), 1226-1257. doi: 10.3934/mcrf.2022036.
[41]	H. J. Sussmann, A product expansion for the Chen series, Theory and Applications of Nonlinear Control Systems, (1986), 323-335.
[42]	P. Tabuada and B. Gharesifard, Universal approximation power of deep residual neural networks via nonlinear control theory, ICLR, (2021).
[43]	D. Thompson, On Growth and Form, Cambridge University Press, 1917.
[44]	F. Vialard, R. Kwitt, S. Wei and M. Niethammer, A shooting formulation of deep learning, Advances in Neural Information Processing Systems, 33 (2020), 11828-11838.
[45]	F.-X. Vialard, L. Risser, D. Rueckert and C. J. Cotter, Diffeomorphic 3D image registration via geodesic shooting using an efficient adjoint calculation, Int. J. Comput. Vis., 97 (2011), 229-241. doi: 10.1007/s11263-011-0481-8.
[46]	L. Younes, Shapes and Diffeomorphisms, Appl. Math. Sci., 171. Springer, 2010.
[47]	L. Younes, Diffeomorphic learning, J. Mach. Learn. Res., 21 (2020), Paper No. 220, 28 pp.
[48]	L. Younes, B. Gris and A. Trouvé, Sub-riemannian methods in shape analysis, Handbook of Variational Methods for Nonlinear Geometric Data, (2020), 463-495. doi: 10.1007/978-3-030-31351-7_17.