Pseudo-Hamiltonian system identification

Sigurd Holmsen; Sølve Eidnes; Signe Riemer-Sørensen

doi:10.3934/jcd.2024001

Article Contents

2024, Volume 11, Issue 1: 59-91. Doi: 10.3934/jcd.2024001

This issue Previous Article Geometric integration on symmetric spaces Next Article B-stability of numerical integrators on Riemannian manifolds

Pseudo-Hamiltonian system identification

1.
Norges Bank, 0107 Oslo, Norway
2.
Department of Mathematics and Cybernetics, SINTEF Digital, 0373 Oslo, Norway

^*Corresponding author
^*Corresponding author

Received: May 2023

Revised: December 2023

Early access: January 8, 2024

Published online: January 8, 2024

This research was supported by the Research Council of Norway and the industry partners Elkem, Eramet Norway, Equinor, BP, Subsea7, Kongsberg Maritime, Aker Solutions and Veas, through the projects BigDataMine (project no. 309691), PRAI (Prediction of Riser-response by Artificial Intelligence) (project no. 308832) and PhysML: Structure-based machine learning for physical systems (project no. 338779).

Abstract / Introduction Full Text(HTML) Figure(21) / Table(7) Related Papers Cited by

Abstract

Identifying the underlying dynamics of physical systems can be challenging when only provided with observational data. In this work, we consider systems that can be modelled as first-order ordinary differential equations. By assuming a certain pseudo-Hamiltonian formulation, we are able to learn the analytic terms of internal dynamics even if the model is trained on data where the system is affected by unknown damping and external disturbances. In cases where it is difficult to find analytic terms for the disturbances, a hybrid model that uses a neural network to learn these can still accurately identify the dynamics of the system as if under ideal conditions. This makes the models applicable in some situations where other system identification models fail. Furthermore, we propose to use a fourth-order symmetric integration scheme in the loss function and avoid actual integration in the training, and demonstrate on varied examples how this leads to increased performance on noisy data.

Keywords:

Mathematics Subject Classification: Primary: 34A55, 37M10; Secondary: 37J99.

Citation:

Full Text(HTML)

Figure 1. Comparison of simulated trajectories for the Hénon–Heiles system, with initial value $ (q,p) = (0.1, -0.2, 0.4, 0.5) $, from $ t = 0 $ to $ t = 10 $

Download: Full-size image PowerPoint slide

Figure 2. The average $ L_2 $-error of the trajectories obtained with 10 different random initial conditions, from the different models of the Hénon–Heiles system trained on noisy data with $ \sigma = 0.02 $, compared to trajectories simulated from the exact system,

Download: Full-size image PowerPoint slide

Figure 3. Phase portraits showing the trained models' trajectories next to the ground truth trajectory, for the nonlinear Schrödinger system. All models are trained on the two training sets, one clean data set and one noisy with $ \sigma = 10^{-4} $. The initial values are $ (q,p) = (-0.3, 0.5, -0.2, -0.4) $

Download: Full-size image PowerPoint slide

Figure 4. The trained coefficients of the PHSI, BSI, and SINDy models of the nonlinear Schrödinger system, plotted against the respective true values. A perfectly trained model will only have points along the dotted line. The models are trained on the noisy data set. Note that PHSI learns the Hamiltonian function while SINDy learns the right-hand side $ g $ of (1) and hence they will not learn the same coefficients for corresponding terms

Download: Full-size image PowerPoint slide

Figure 5. Comparison between the phase portraits obtained from integrating the exact forced and damped mass-spring system and the learned models from time 0 to 10. The initial value is $ (q,p) = (-3.4, -1.9) $

Download: Full-size image PowerPoint slide

Figure 6. Average $ L_2 $-error of 30 simulated trajectories of the forced and damped mass-spring system, with random initial conditions

Download: Full-size image PowerPoint slide

Figure 7. Simulations of the tank system. Left: The volume of the fluid in the leaky fourth tank simulated from the exact system and the different models. Right: The leak approximated by the neural network in the hybrid PHSI model, compared to the exact solution. The upper plots have initial conditions within the distribution of the training data: $ (\phi,\mu) = (-0.4, 0, 0.5, 0, 0.2, 0 -0.6, -0.5, 0.5) $. The lower plots show extrapolation in time and space; time from 0 to 1 and initial state values $ (\phi,\mu) = (10, 19, 4, 19, 7, 9, 17, 9, 11) $

Download: Full-size image PowerPoint slide

Figure 8. Average $ L_2 $-error of 30 sets of simulated future tank volumes and pipe flows in the tank system, trained on noisy data with $ \sigma = 0.005 $

Download: Full-size image PowerPoint slide

Figure 9. The mean $ L_2 $ error of PHSI models trained with the different integrators on the Hénon--Heiles problem. The error is on the predicted positions and momenta from $ t = 0 $ to $ t = 10 $ on 10 different random initial conditions

Download: Full-size image PowerPoint slide

Figure 10. Prediction of $ q_1 $ by PHSI models trained with different integrators on the Hénon--Heiles problem. The initial condition is $ q = (-0.2, 0.2) $, $ p = (0.1,-0.2) $

Download: Full-size image PowerPoint slide

Figure 11. The mean $ L_2 $ error of hybrid PHSI models trained with the different integrators on the tank system. The error is of the predicted volume and flow in all tanks and pipes from $ t = 0 $ to $ t = 1 $ on 10 different random initial conditions

Download: Full-size image PowerPoint slide

Figure 12. Volume of the fourth tank as predicted by hybrid PHSI models trained with different integrators on the tank system. The initial condition is $ \phi^0 = (-1, -1, 0, \frac{1}{2}, -1) $, $ \mu^0 = (1, 1, -\frac{1}{2}, -1) $

Download: Full-size image PowerPoint slide

Figure 13. Average $ L_2 $-error over 30 simulated trajectories of the Hénon–Heiles system by PHSI models trained with different combinations of regularization parameter $ \lambda_H $ and pruning parameter $ P $. The models are trained for 3 epochs, which means that when $ P = 4 $, there is no pruning

Download: Full-size image PowerPoint slide

Figure 14. Simulated trajectories obtained for different PHSI models and the exact solution, for the nonlinear Hénon–Heiles system for three different combinations of regularization and pruning

Download: Full-size image PowerPoint slide

Figure 15. Average $ L_2 $-error over 30 simulated trajectories of the nonlinear Schrödinger system by PHSI models trained with different combinations of regularization parameter $ \lambda_H $ and pruning parameter $ P $. The models are trained for 10 epochs, meaning when $ P = 10 $, the pruning only occurs once

Download: Full-size image PowerPoint slide

Figure 16. Simulated PHSI trajectories along with a simulation of the exact nonlinear Schödinger system, for three different combinations of regularization and pruning

Download: Full-size image PowerPoint slide

Figure 17. Average $ L_2 $-error over 30 simulated trajectories of the damped mass-spring system by PHSI models trained with different combinations of regularization-parameter $ \lambda_H $ and pruning-parameter $ P $. The models are trained for 80 epochs, so that when $ P = 80 $, pruning is only done after the last epoch

Download: Full-size image PowerPoint slide

Figure 18. Simulated PHSI trajectories along with a simulation of the exact damped mass-spring system, for three different combinations of regularization and pruning

Download: Full-size image PowerPoint slide

Figure 19. Average loss over simulated trajectories for PHSI models trained with different combinations of regularization-parameter $ \lambda_H $ and pruning-parameter $ P $. The models are trained for 80 epochs, which means that when $ P = 80 $, pruning is only done after the last epoch

Download: Full-size image PowerPoint slide

Figure 20. Simulated PHSI trajectories along with the exact solution of one of the leaking tanks in the connected tank system for three different combinations of regularization and pruning

Download: Full-size image PowerPoint slide

Figure 21. Phase plots of trajectories $ x = (q,p) $ obtained from systems where $ \hat{\dot{x}} $ is given by different combinations of $ S(x)\nabla H $ and $ f $. Each column is a different combination of regularization parameters, where $ \lambda_1 $ and $ \lambda_2 $ are the magnitudes of $ l_1 $ regularization on $ \hat{H}_\theta $ and $ \hat{F}_\theta $, respectively

Download: Full-size image PowerPoint slide

Table 1. Learned coefficients in the Hamiltonian for the Hénon–Heiles system

	$ q_1^2 $	$ q_2^2 $	$ q_1^2 q_2 $	$ q_2^3 $	$ p_1^2 $	$ p_2^2 $
True Value	$ 0.5 $	$ 0.5 $	1	$ -0.333 $	$ 0.5 $	$ 0.5 $
PHSI	$ 0.509 $	$ 0.495 $	$ 1.009 $	$ -0.338 $	$ 0.501 $	$ 0.477 $
SSINN	$ 0.421 $	$ 0.378 $	$ 0.569 $	$ -0.195 $	$ 0.484 $	$ 0.443 $
	$ q_2 p_1^2 $	$ q_2 $	$ q_1q_2 $	$ p_2 $	$ p_1p_2 $
True Value	0	0	0	0	0
PHSI	$ 0.124 $	0	0	0	0
SSINN	N/A	$ 0.004 $	$ 0.002 $	$ -0.006 $	$ -0.006 $

| Show Table

DownLoad: CSV

Table 2. Learned coefficients for the forced and damped mass-spring problem. $ q $ and $ p $ are multiplied with the trained coefficients while $ c, \alpha, \omega $ and $ const. $ are themselves the trainable parameters. Empty entries mean that the model does not learn that term

Trained PHSI parameters
	$ q^2 $	$ p^2 $	$ c $	$ \alpha $	$ \omega $
True value	$ 0.5 $	$ 0.5 $	$ 0.3 $	2	$ 0.5 $
$ \hat{H} $	$ 0.473 $	$ 0.484 $
$ \hat{R} $			0.300
$ \hat{F} $	0	0	0	1.984	0.505
Trained BSI and SINDy parameters
	$ q $	$ p $	$ const. $	$ \alpha $	$ \omega $
True $ \hat{\dot{q}} $	0	1	0	0	0
BSI $ \hat{\dot{q}} $	0	0.991	0	0	0
SINDy $ \hat{\dot{q}} $	0	0.983	0	0	0
True $ \hat{\dot{p}} $	$ -1 $	$ -0.3 $	0	2	$ 0.5 $
BSI $ \hat{\dot{p}} $	$ -0.980 $	$ -0.278 $	0	$ 1.999 $	$ 0.506 $
SINDy $ \hat{\dot{p}} $	$ -0.548 $	0	$ 0.241 $

| Show Table

DownLoad: CSV

Table 3. Learned coefficients for the tank system on the noisy data. $ x_1^2, \dots, x_9^2 $ are multiplied with the trained coefficients while $ r_1, \dots, r_5 $ are themselves the trainable parameters

	$ x_1^2 $	$ x_2^2 $	$ x_3^2 $	$ x_4^2 $	$ x_5^2 $	$ x_6^2 $	$ x_7^2 $
True value	25	25	25	25	25	$ 4.905 $	$ 4.905 $
PHSI	$ 24.94 $	$ 24.98 $	$ 24.98 $	$ 24.95 $	$ 24.97 $	$ 4.930 $	$ 4.960 $
	$ x_8^2 $	$ x_9^2 $	$ r_1 $	$ r_2 $	$ r_3 $	$ r_4 $	$ r_5 $
True value	$ 4.905 $	$ 4.905 $	$ 0.03 $	$ 0.03 $	$ 0.09 $	$ 0.05 $	$ 0.05 $
PHSI	$ 4.890 $	$ 4.930 $	$ 0.031 $	$ 0.029 $	$ 0.086 $	$ 0.051 $	$ 0.041 $

| Show Table

DownLoad: CSV

Table 4. The learned approximations of $ \gamma $ in (14), for different $ \gamma $ and different pruning tresholds $ \epsilon $, trained on noise-free data

$ \epsilon \backslash \gamma $	$ 0.1 $	$ 0.07 $	$ 0.05 $	$ 0.03 $	$ 0.01 $	$ 0.007 $	$ 0.005 $	$ 0.003 $	$ 0.001 $
$ 0.05 $	$ 0.1012 $	$ 0.0718 $	0	0	0	0	0	0	0
$ 0.01 $	$ 0.1000 $	$ 0.0717 $	$ 0.0501 $	$ 0.0310 $	0	0	0	0	0
$ 0.001 $	$ 0.1006 $	$ 0.0711 $	$ 0.0489 $	$ 0.0301 $	$ 0.0102 $	$ 0.0051 $	0	$ 0.0028 $	0

| Show Table

DownLoad: CSV

Table 5. The learned approximations of $ \gamma $ in (14), for different $ \gamma $ and different pruning tresholds $ \epsilon $, trained on data with added Gaussian noise with standard deviation $ \sigma = 0.2 $

$ \epsilon \backslash \gamma $	$ 0.1 $	$ 0.07 $	$ 0.05 $	$ 0.03 $	$ 0.01 $	$ 0.007 $	$ 0.005 $	$ 0.003 $	$ 0.001 $
$ 0.05 $	$ 0.0974 $	0	0	0	0	0	0	0	0
$ 0.01 $	$ 0.1001 $	$ 0.0485 $	$ 0.0515 $	$ 0.0412 $	0	0	0	0	0
$ 0.001 $	$ 0.0833 $	$ 0.0754 $	$ 0.0372 $	$ 0.0399 $	$ 0.0034 $	$ 0.0093 $	0	$ -0.0196 $	$ -0.0062 $

| Show Table

DownLoad: CSV

Table 6. Properties of integrators. PRK4 is explicit, mono-implicit and symplectic for separable systems but not for non-separable systems

Integrator	order	$ g $ eval's	explicit	mono-implicit	symmetric	symplectic
Euler	1	1	yes	yes	no	no
Midpoint	2	1	no	yes	yes	yes
RK4	4	4	yes	yes	no	no
SRK4	4	4	no	yes	yes	no
SRK6	6	5	no	yes	yes	no
PRK4	4	7	yes/no	yes/no	yes	yes/no

| Show Table

DownLoad: CSV

Table 7. Mean and standard deviation of the predicted friction coefficients of the tank system, relative to the ground truth $ R_p = (0.03, 0.03, 0.09, 0.03, 0.03) $ (i.e. so that 1 corresponds to the correct coefficient)

	no noise	$ \sigma=0.03 $	$ \sigma=0.05 $
7500 training points
Euler	$ 22.75\pm7.20 $	$ 23.59\pm7.04 $	$ 25.27\pm7.31 $
Midpoint	$ 1.27\pm0.04 $	$ 1.16\pm0.14 $	$ 1.01\pm0.11 $
RK4	$ 1.04\pm0.04 $	$ 1.70\pm0.24 $	$ 1.95\pm0.27 $
SRK4	$ 1.16\pm0.07 $	$ 0.92\pm0.13 $	$ 0.88\pm0.43 $
SRK6	$ 0.95\pm0.07 $	$ 0.85\pm0.19 $	$ 0.94\pm0.25 $
30000 training points
Euler	$ 11.33\pm3.31 $	$ 12.31\pm3.48 $	$ 13.83\pm3.91 $
Midpoint	$ 1.23\pm0.11 $	$ 1.15\pm0.16 $	$ 1.16\pm0.25 $
RK4	$ 1.20\pm0.06 $	$ 1.97\pm0.29 $	$ 4.18\pm0.70 $
SRK4	$ 1.16\pm0.04 $	$ 1.11\pm0.13 $	$ 1.00\pm0.23 $
SRK6	$ 1.24\pm0.06 $	$ 1.21\pm0.10 $	$ 1.35\pm0.32 $

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	F. Alet, D. Doblar, A. Zhou, J. Tenenbaum, K. Kawaguchi and C. Finn, Noether networks: Meta-learning useful conserved quantities, Advances in Neural Information Processing Systems, 34 (2021), 16384-16397.
[2]	V. I. Arnold, Mathematical Methods of Classical Mechanics, volume 60 of Graduate Texts in Mathematics, Springer-Verlag, New York, second edition, 1989. Translated from the Russian by K. Vogtmann and A. Weinstein. doi: 10.1007/978-1-4757-2063-1.
[3]	P. Benner, P. Goyal and P. Van Dooren, Identification of port-Hamiltonian systems from frequency response data, Systems Control Lett., 143 2020, 104741, 10 pp. doi: 10.1016/j.sysconle.2020.104741.
[4]	S. L. Brunton, J. L. Proctor and J. N. Kutz, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proceedings of the National Academy of Sciences, 113 (2016), 3932-3937. doi: 10.1073/pnas.1517384113.
[5]	J. R. Cash and A. Singhal, High order methods for the numerical solution of two-point boundary value problems, BIT, 22 (1982), 184-199. doi: 10.1007/BF01944476.
[6]	J. R. Cash and A. Singhal, Mono-implicit Runge-Kutta formulae for the numerical integration of stiff differential systems, IMA J. Numer. Anal., 2 (1982), 211-227. doi: 10.1093/imanum/2.2.211.
[7]	T. Chen, Y. Rubanova, J. Bettencourt and D. K. Duvenaud, Neural ordinary differential equations, Advances in Neural Information Processing Systems, 31 (2018), 6572-6583.
[8]	Z. Chen, J. Zhang, M. Arjovsky and L. Bottou, Symplectic recurrent neural networks, in International Conference on Learning Representations, 2020.
[9]	K. Cherifi, P. Goyal and P. Benner, A non-intrusive method to inferring linear port-Hamiltonian realizations using time-domain data, Electron. Trans. Numer. Anal., 56 (2022), 102-116. doi: 10.1553/etna_vol56s102.
[10]	M. Cranmer, S. Greydanus, S. Hoyer, P. Battaglia, D Spergel and S. Ho, Lagrangian neural networks, ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations, 2020.
[11]	M. Cranmer, Al. Sanchez Gonzalez, P. Battaglia, R. Xu, K. Cranmer, D. Spergel and S. Ho, Discovering symbolic models from deep learning with inductive biases, Advances in Neural Information Processing Systems, 33 (2020), 17429-17442.
[12]	G. Cybenko, Approximation by superpositions of a sigmoidal function, Mathematics of Control, Signals and Systems, 2 (1989), 303-314. doi: 10.1007/BF02551274.
[13]	M. David and F. Méhats, Symplectic learning for Hamiltonian neural networks, J. Comput. Phys., 494 (2023), Paper No. 112495, 13 pp. doi: 10.1016/j.jcp.2023.112495.
[14]	S. A. Desai, M. Mattheakis and S. J. Roberts, Variational integrator graph networks for learning energy-conserving dynamical systems, Physical Review E, 104 (2021), Paper No. 035310, 9 pp. doi: 10.1103/physreve.104.035310.
[15]	S. A. Desai, M. Mattheakis, D. Sondak, P. Protopapas and S. J. Roberts, Port-Hamiltonian neural networks for learning explicit time-dependent dynamical systems, Phys. Rev. E, 104 (2021), 034312. doi: 10.1103/PhysRevE.104.034312.
[16]	D. DiPietro, S. Xiong and B. Zhu, Sparse symplectically integrated neural networks, Advances in Neural Information Processing Systems, 33 (2020), 6074-6085.
[17]	T. Duong and N. Atanasov, Hamiltonian-based neural ODE networks on the SE(3) manifold for dynamics learning and control, in Robotics: Science and Systems (RSS), 2021. doi: 10.15607/RSS.2021.XVII.086.
[18]	T. Duong and N. Atanasov, Adaptive control of SE(3) Hamiltonian dynamics with learned disturbance features, IEEE Control Syst. Lett., 6 (2022), 2773-2778. doi: 10.1109/LCSYS.2022.3177156.
[19]	S. Eidnes and K. O. Lye, Pseudo-Hamiltonian neural networks for learning partial differential equations, Journal of Computational Physics, 2024, 112738. doi: 10.1016/j.jcp.2023.112738.
[20]	S. Eidnes, A. J. Stasik, C. Sterud, E. Bøhn and S. Riemer-Sørensen, Pseudo-Hamiltonian neural networks with state-dependent external forces, Phys. D, 446 (2023), Paper No. 133673, 13 pp. doi: 10.1016/j.physd.2023.133673.
[21]	U. Fasel, J. N. Kutz, B. W. Brunton and S. L. Brunton, Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control, Proceedings of the Royal Society A, 478 (2022), Paper No. 20210904, 20 pp. doi: 10.1098/rspa.2021.0904.
[22]	M. Finzi, K. A. Wang and and A. G. Wilson, Simplifying Hamiltonian and Lagrangian neural networks via explicit constraints, Advances in Neural Information Processing Systems, 33 (2020), 13880-13889.
[23]	S. Greydanus, M. Dzamba and J. Yosinski, Hamiltonian neural networks, in H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32, Curran Associates, Inc., 2019.
[24]	E. Hairer, C. Lubich and G. Wanner, Geometric Numerical Integration, volume 31 of Springer Series in Computational Mathematics, Springer-Verlag, Berlin, second edition, 2006. Structure-preserving algorithms for ordinary differential equations.
[25]	M. Hénon and C. Heiles, The applicability of the third integral of motion: Some numerical experiments, Astronom. J., 69 (1964), 73-79. doi: 10.1086/109234.
[26]	K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks, 4 (1991), 251-257. doi: 10.1016/0893-6080(91)90009-T.
[27]	P. Jin, Z. Zhang, I. G. Kevrekidis and G. E. Karniadakis, Learning Poisson systems and trajectories of autonomous systems via Poisson neural networks, IEEE Transactions on Neural Networks and Learning Systems, 34 (2023), 8271-8283. doi: 10.1109/TNNLS.2022.3148734.
[28]	P. Jin, Z. Zhang, A. Zhu, Y. Tang and G. E. Karniadakis, SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems, Neural Networks, 132 (2020), 166-179. doi: 10.1016/j.neunet.2020.08.017.
[29]	K. Kaheman, J. N. Kutz and S. L. Brunton, SINDy-PI: A robust algorithm for parallel implicit sparse identification of nonlinear dynamics, Proceedings of the Royal Society A, 476 (2020), 20200279, 25 pp. doi: 10.1098/rspa.2020.0279.
[30]	E. Kaiser, J. N. Kutz and S. L. Brunton, Sparse identification of nonlinear dynamics for model predictive control in the low-data limit, Proceedings of the Royal Society A, 474 (2018), 20180335, 25 pp. doi: 10.1098/rspa.2018.0335.
[31]	G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang and L. Yang, Physics-informed machine learning, Nature Reviews Physics, 3 (2021), 422-440. doi: 10.1038/s42254-021-00314-5.
[32]	S. Klus, F. Nüske, S. Peitz, J.-H. Niemann, C. Clementi and C. Schütte, Data-driven approximation of the Koopman generator: Model reduction, system identification, and control, Phys. D, 406 (2020), 132416, 15 pp. doi: 10.1016/j.physd.2020.132416.
[33]	J. R. Koza, Genetic programming as a means for programming computers by natural selection, Statistics and Computing, 4 (1994), 87-112. doi: 10.1007/BF00175355.
[34]	K. Lee, N. Trask and P. Stinis, Structure-preserving sparse identification of nonlinear dynamics for data-driven modeling, Proceedings of Mathematical and Scientific Machine Learning, 190 (2022), 65-80.
[35]	Z. Liu and M. Tegmark, Machine learning conservation laws from trajectories, Phys. Rev. Lett., 126 (2021), Paper No. 180604, 6 pp. doi: 10.1103/PhysRevLett.126.180604.
[36]	L. Ljung, System Identification: Theory for the User, Prentice Hall Information and System Sciences Series. Prentice Hall, Inc., Englewood Cliffs, NJ, 1987.
[37]	Z. Long, Y. Lu and B. Dong, PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network, J. Comput. Phys., 399 (2019), 108925, 17 pp. doi: 10.1016/j.jcp.2019.108925.
[38]	J. E. Marsden and T. S. Ratiu, Introduction to Mechanics and Symmetry, volume 17 of Texts in Applied Mathematics, Springer-Verlag, New York, second edition, 1999. A basic exposition of classical mechanical systems. doi: 10.1007/978-0-387-21792-5.
[39]	T. Matsubara, A. Ishikawa and T. Yaguchi, Deep energy-based modeling of discrete-time physics, in H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan and H. Lin, editors, Advances in Neural Information Processing Systems, 33 (2020), 13100-13111. Curran Associates, Inc..
[40]	T. Matsubara and T. Yaguchi, Finde: Neural differential equations for finding and preserving invariant quantities, arXiv preprint, arXiv: 2210.00272, 2022.
[41]	R. I. McLachlan, G. R. W. Quispel and N. Robidoux, Geometric integration using discrete gradients, R. Soc. Lond. Philos. Trans. Ser. A Math. Phys. Eng. Sci., 357 (1999), 1021-1045. doi: 10.1098/rsta.1999.0363.
[42]	R. Morandin, J. Nicodemus and B. Unger, Port-Hamiltonian dynamic mode decomposition, SIAM J. Sci. Comput., 45 (2023), A1690-A1710. doi: 10.1137/22M149329X.
[43]	H. Noren, Learning Hamiltonian systems with mono-implicit Runge–Kutta methods, Geometric science of information. Part Ⅰ. Lecture Notes in Comput. Sci., Springer, Cham, 14071 (2023), 552-559. doi: 10.1007/978-3-031-38271-0_55.
[44]	C. Offen and S. Ober-Blöbaum, Symplectic integration of learned Hamiltonian systems, Chaos, 32 (2022), Paper No. 013122, 13 pp. doi: 10.1063/5.0065913.
[45]	C. Rackauckas, Y. Ma, J. Martensen, C. Warner, K. Zubov, R. Supekar, D. Skinner, A. Ramadhan and A. Edelman, Universal differential equations for scientific machine learning, arXiv preprint, arXiv: 2001.04385, 2021. doi: 10.21203/rs.3.rs-55125/v1.
[46]	M. Raissi, P. Perdikaris and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378 (2019), 686-707. doi: 10.1016/j.jcp.2018.10.045.
[47]	S. H Rudy, S. L. Brunton, J. L. Proctor and J. N. Kutz, Data-driven discovery of partial differential equations, Science Advances, 3 (2017), e1602614. doi: 10.1126/sciadv.1602614.
[48]	H. Schaeffer, Learning partial differential equations via data discovery and sparse optimization, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473 (2017), 20160446, 20 pp. doi: 10.1098/rspa.2016.0446.
[49]	M. Schmidt and H. Lipson, Distilling free-form natural laws from experimental data, Science, 324 (2009), 81-85. doi: 10.1126/science.1165893.
[50]	M. Tao, Explicit symplectic approximation of nonseparable Hamiltonians: Algorithm and long time performance, Physical Review E, 94 (2016), 043303, 10 pp. doi: 10.1103/physreve.94.043303.
[51]	S.-M. Udrescu and M. Tegmark, AI feynman: A physics-inspired method for symbolic regression, Science Advances, 6 (2020), eaay2631. doi: 10.1126/sciadv.aay2631.
[52]	W. M. G. van Bokhoven, Efficient higher order implicit one-step methods for integration of stiff differential equations, BIT, 20 (1980), 34-43. doi: 10.1007/BF01933583.
[53]	A. van der Schaft, Port-Hamiltonian systems: An introductory survey, in Proceedings of the International Congress of Mathematicians, volume 3, (2006), 1339-1365. Citeseer. doi: 10.4171/022-3/65.
[54]	A. van der Schaft and D. Jeltsema, Port-Hamiltonian systems theory: An introductory overview, Foundations and Trends in Systems and Control, 1 (2014), 173-378. doi: 10.1561/2600000002.
[55]	A. van der Schaft and B. Maschke, The Hamiltonian formulation of energy conserving physical systems with external ports, AEU. Archiv für Elektronik und Übertragungstechnik, 49 (1995), 362-371.
[56]	J. Willard, X. Jia, S. Xu, M. Steinbach and V. Kumar, Integrating scientific knowledge with machine learning for engineering and environmental systems, ACM Comput. Surv., 55 (2022), 1-37. doi: 10.1145/3514228.
[57]	H. Yoshida, Construction of higher order symplectic integrators, Phys. Lett. A, 150 (1990), 262-268. doi: 10.1016/0375-9601(90)90092-3.
[58]	Z. Zhang, Y. Shin and G. E. Karniadakis, GFINNs: GENERIC formalism informed neural networks for deterministic and stochastic dynamical systems, Philos. Trans. Roy. Soc. A, 380 (2022), Paper No. 20210207, 21 pp. doi: 10.1098/rsta.2021.0207.
[59]	Y. D. Zhong, B. Dey and A. Chakraborty, Dissipative SymODEN: Encoding Hamiltonian dynamics with dissipation and control into deep learning, in ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations, 2020.
[60]	Y. D. Zhong, B. Dey and A. Chakraborty, Symplectic ODE-net: Learning Hamiltonian dynamics with control, in International Conference on Learning Representations, 2020.
[61]	A. Zhu, P. Jin and Y. Tang, Deep Hamiltonian networks based on symplectic integrators, arXiv preprint, arXiv: 2004.13830, 2020.
[62]	A. Zhu, P. Jin, B. Zhu and Y. Tang, On numerical integration in neural ordinary differential equations, in International Conference on Machine Learning, (2022), 27527-27547. PMLR.