2019

Symplectic Recurrent Neural Networks

Chen, Zhengdao, Zhang, Jianyu, Arjovsky, Martin et al.

Understand

We propose Symplectic Recurrent Neural Networks (SRNNs) as learning algorithms that capture the dynamics of physical systems from observed trajectories.

An SRNN models the Hamiltonian function of the system by a neural network and furthermore leverages symplectic integration, multiple-step training and initial state optimization to address the challenging numerical issues associated with Hamiltonian systems.
We show that SRNNs succeed reliably on complex and noisy Hamiltonian systems.
We also show how to augment the SRNN integration scheme in order to handle stiff dynamical systems such as bouncing billiards.

Built on

Recurrent neural networks in the eye of differential equations
Original
Niu, M. Y., Horesh, L., and Chuang, I. (2019) · 1904
Earlier work this paper cites.
Hamiltonian neural networks
Original
Greydanus, S., Dzamba, M., and Yosinski, J. (2019) · 1906
Earlier work this paper cites.
Hamiltonian graph networks with ode integrators
Original
Sanchez-Gonzalez, A., Bapst, V., Cranmer, K., and Battaglia, P. (2019) · 1909
Earlier work this paper cites.
Numerical Methods for Ordinary Differential Systems: The Initial Value Problem
Lambert, J. D. (1991) · 1991
Earlier work this paper cites.
Backward analysis of numerical integrators and symplectic methods
Hairer, E. (1994) · 1994
Earlier work this paper cites.
Long short-term memory
Hochreiter, S. and Schmidhuber, J. (1997) · 1997
Earlier work this paper cites.
Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization
Zhu, C., Byrd, R. H., Lu, P., and Nocedal, J. (1997) · 1997
Earlier work this paper cites.
Geometric Numerical Integration: Structure-Preserving Algorithms for Ordinary Differential Equations
Hairer, E., Lubich, C., and Wanner, G. (2002) · 2002
Earlier work this paper cites.

Similar

On the nonlinear stability of symplectic integrators
McLachlan, R. I., Perlmutter, M., and Quispel, G. R. W. (2004) · 2004
Cited alongside, same era.
Simulating Hamiltonian Dynamics
Leimkuhler, B. and Reich, S. (2005) · 2005
Cited alongside, same era.
Parameter estimation in ordinary differential equations for biochemical processes using the method of multiple shooting
Peifer, M. and Timmer, J. (2007) · 2007
Cited alongside, same era.
Solving Ordinary Differential Equations I: Nonstiff Problems
Hairer, E., Nørsett, S. P., and Wanner, G. (2008) · 2008
Cited alongside, same era.
Solving Ordinary Differential Equations II: Stiff and Differential - Algebraic Problems
Hairer, E. and Wanner, G. (2013) · 2013
Cited alongside, same era.
Adam: A method for stochastic optimization
Original
Kingma, D. P. and Ba, J. (2014) · 2014
Cited alongside, same era.
Unitary evolution recurrent neural networks
Original
Arjovsky, M., Shah, A., and Bengio, Y. (2015) · 2015
Cited alongside, same era.
Neural ordinary differential equations
Chen, T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D. K. (2018a)
Cited in the paper.
Structure-preserving numerical integrators for hodgkin-huxley-type systems
Chen, Z., Raman, B., and Stern, A. (2018b)
Cited in the paper.

Then

Interaction networks for learning about objects, relations and physics
Original
Battaglia, P. W., Pascanu, R., Lai, M., Rezende, D. J., and Kavukcuoglu, K. (2016) · 2016
Later among the works it cites.
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
Brunton, S. L., Proctor, J. L., and Kutz, J. N. (2016) · 2016
Later among the works it cites.
A compositional object-based approach to learning physical dynamics
Original
Chang, M. B., Ullman, T., Torralba, A., and Tenenbaum, J. B. (2016) · 2016
Later among the works it cites.
Explicit symplectic approximation of nonseparable hamiltonians: Algorithm and long time performance
Tao, M. (2016) · 2016
Later among the works it cites.
Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis
Stapor, P., Fröhlich, F., and Hasenauer, J. (2018) · 2018
Later among the works it cites.
AntisymmetricRNN: A dynamical system view on recurrent neural networks
Chang, B., Chen, M., Haber, E., and Chi, E. H. (2019) · 2019
Closest in time.
Deep lagrangian networks: Using physics as model prior for deep learning
Lutter, M., Ritter, C., and Peters, J. (2019) · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…