Hamiltonian Neural Networks

Hamiltonian Neural Networks, introduced in 2019 by Greydanus et al. in this paper¹, build upon the idea of adding physics-informed soft constraints in the loss function of the neural network (based on Raissi et al. paper²).

Physics Insight

Hamiltonian neural networks target a class of physical systems called Hamiltonian systems, which are endowed by a specific structure.

Architecture

The Hamiltonian neural network (HNN) aims to approximate the Hamiltonian quantity $H$ by using a feedforward neural networks whose objective is to match Hamilton's canonical equations, so as to preserve the "structure" of the Hamiltonian system.

The authors generate a dataset of position and momentum values (along with their derivatives) by using the fourth-order Runge-Kutta integrator. The loss function used in HNN is:
$L = \left\lVert \dfrac{\partial \hat{\mathcal{H_{\theta}}}}{\partial \mathbf{p}} - \dfrac{d \mathbf{q}}{d t} \right\rVert ^2 + \left\lVert \dfrac{\partial \hat{\mathcal{H_{\theta}}}}{\partial \mathbf{q}} + \dfrac{d \mathbf{p}}{d t} \right\rVert ^2,$
where the $\dfrac{d \mathbf{q}}{dt}$ and $\dfrac{d \mathbf{p}}{dt}$ are calculated in the data generation phase (by finding slope of the $q(t)$ and $p(t)$ graphs using finite-difference) and the $\dfrac{\partial \hat{\mathcal{H_{\theta}}}}{\partial \mathbf{q}}$ and $\dfrac{\partial \hat{\mathcal{H_{\theta}}}}{\partial \mathbf{p}}$ are computed using auto-differentiation³ of the output of the HNN with respect to the input.

A Few Notes

• The use of a non-structure preserving integrator has an effect on the accuracy and validity of the results. This deficiency led to the introduction of Symplectic Recurrent Neural Networks.