2020

Scalable Gradients for Stochastic Differential Equations

Li, Xuechen, Wong, Ting-Kam Leonard, Chen, Ricky T. Q. et al.

Understand

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations.

We generalize this method to stochastic differential equations, allowing time-efficient and constant-memory computation of gradients with high-order adaptive solvers.
Specifically, we derive a stochastic differential equation whose solution is the gradient, a memory-efficient algorithm for caching noise, and conditions under which numerical solutions converge.
In addition, we combine our method with gradient-based stochastic variational inference for latent stochastic differential equations.

Built on

Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit
Original
Belinda Tzen and Maxim Raginsky · 1905
Earlier work this paper cites.
Computational aspects of maximum likelihood estimation and reduction in sensitivity function calculations
Narendra Gupta and Raman Mehra · 1974
Earlier work this paper cites.
Sobolev Spaces
R Adams · 1975
Earlier work this paper cites.
Ordinary Differential Equations
VI Arnold · 1978
Earlier work this paper cites.
Learning representations by back-propagating errors
David E Rumelhart, Geoffrey E Hinton, Ronald J Williams, et al · 1988
Earlier work this paper cites.
A generalized itô-ventzell formula. application to a class of anticipating stochastic differential equations
Daniel Ocone and Étienne Pardoux · 1989
Earlier work this paper cites.
Likelihood ratio gradient estimation for stochastic systems
Peter W Glynn · 1990
Earlier work this paper cites.
A general stochastic maximum principle for optimal control problems
Shige Peng · 1990
Earlier work this paper cites.
A monte carlo method for sensitivity analysis and parametric optimization of nonlinear stochastic systems
Jichuan Yang and Harold J Kushner · 1991
Earlier work this paper cites.
Some guidelines and guarantees for common random numbers
Paul Glasserman and David D Yao · 1992
Earlier work this paper cites.
Backward stochastic differential equations and quasilinear parabolic partial differential equations
Etienne Pardoux and Shige Peng · 1992
Earlier work this paper cites.
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Ronald J Williams · 1992
Earlier work this paper cites.
On the convergence rates of ipa and fdc derivative estimators
Pierre L’Ecuyer and Gaétan Perron · 1994
Earlier work this paper cites.
Numerical integration of stochastic differential equations , volume 313
Grigorii Noikhovich Milstein · 1994
Earlier work this paper cites.
Gradient calculations for dynamic recurrent neural networks: A survey
Barak A Pearlmutter · 1995
Earlier work this paper cites.
Linear gaussian state space modeling
Genshiro Kitagawa and Will Gersch · 1996
Earlier work this paper cites.
Optimization and sensitivity analysis of computer simulation models by the score function method
Jack PC Kleijnen and Reuven Y Rubinstein · 1996
Earlier work this paper cites.
Variable step size control in the numerical solution of stochastic differential equations
Jessica G Gaines and Terry J Lyons · 1997
Earlier work this paper cites.
Differential equations driven by rough signals
Terry J Lyons · 1998
Earlier work this paper cites.
Fully coupled forward-backward stochastic differential equations and applications to optimal control
Shige Peng and Zhen Wu · 1999
Earlier work this paper cites.
An introduction to numerical methods for stochastic differential equations
Eckhard Platen · 1999
Earlier work this paper cites.
Diffusions, Markov Processes and Martingales: Volume 2, Itô Calculus , volume 2
L Chris G Rogers and David Williams · 2000
Earlier work this paper cites.
Infinite-horizon gradient-based policy search
Jonathan Baxter and Peter L Bartlett · 2001
Earlier work this paper cites.
Joint characteristic function and simultaneous simulation of iterated itô integrals for multiple independent brownian motions
Magnus Wiktorsson et al · 2001
Earlier work this paper cites.
Stochastic Differential Equations
Bernt Øksendal · 2003
Earlier work this paper cites.
Adaptive stepsize based on control theory for stochastic differential equations
Pamela M Burrage, R Herdiana, and Kevin Burrage · 2004
Earlier work this paper cites.
Runge–Kutta methods for stratonovich stochastic differential equation systems with commutative noise
Andreas Rößler · 2004
Earlier work this paper cites.
Stochastic calculus for finance II: Continuous-time models , volume 11
Steven E Shreve · 2004
Earlier work this paper cites.
Sensitivity analysis using Itô–Malliavin calculus and martingales, and application to stochastic optimal control
Emmanuel Gobet and Rémi Munos · 2005
Earlier work this paper cites.
Smoking adjoints: Fast Monte Carlo greeks
Mike Giles and Paul Glasserman · 2006
Earlier work this paper cites.

Similar

The pathwise convergence of approximation schemes for stochastic differential equations
Peter E Kloeden and Andreas Neuenkirch · 2007
Cited alongside, same era.
Gaussian process dynamical models for human motion
Jack M Wang, David J Fleet, and Aaron Hertzmann · 2007
Cited alongside, same era.
Variational inference for diffusion processes
Cédric Archambeau, Manfred Opper, Yuan Shen, Dan Cornford, and John S Shawe-Taylor · 2008
Cited alongside, same era.
Runge–Kutta methods for the strong approximation of solutions of stochastic differential equations
Andreas Rößler · 2010
Cited alongside, same era.
State-space inference and learning with gaussian processes
Ryan Turner, Marc Deisenroth, and Carl Rasmussen · 2010
Cited alongside, same era.
Parallel random numbers: as easy as 1, 2, 3
John K Salmon, Mark A Moraes, Ron O Dror, and David E Shaw · 2011
Cited alongside, same era.
Mathematical population genetics 1: theoretical introduction , volume 27
Warren J Ewens · 2012
Cited alongside, same era.
Normal approximations with Malliavin calculus: from Stein’s method to universality , volume 192
Ivan Nourdin and Giovanni Peccati · 2012
Cited alongside, same era.
A general-purpose software framework for dynamic optimization
Joel Andersson · 2013
Cited alongside, same era.
Splittable pseudorandom number generators using cryptographic hashing
Koen Claessen and Michał H Pałka · 2013
Cited alongside, same era.
Auto-encoding variational bayes
Original
Diederik P Kingma and Max Welling · 2013
Cited alongside, same era.
Numerical solution of stochastic differential equations , volume 23
Peter E Kloeden and Eckhard Platen · 2013
Cited alongside, same era.

Then

Stable architectures for deep neural networks
Eldad Haber and Lars Ruthotto · 2017
Later among the works it cites.
beta-vae: Learning basic visual concepts with a constrained variational framework
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner · 2017
Later among the works it cites.
Structured inference networks for nonlinear state space models
Rahul G Krishnan, Uri Shalit, and David Sontag · 2017
Later among the works it cites.
Maximum principle based algorithms for deep learning
Qianxiao Li, Long Chen, Cheng Tai, and E Weinan · 2017
Later among the works it cites.
Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations
Original
Yiping Lu, Aoxiao Zhong, Quanzheng Li, and Bin Dong · 2017
Later among the works it cites.
Automatic differentiation in pytorch
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer · 2017
Later among the works it cites.
Adaptive methods for stochastic differential equations via natural embeddings and rejection sampling with memory
Christopher Rackauckas and Qing Nie · 2017
Later among the works it cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Later among the works it cites.
A proposal on machine learning via dynamical systems
E Weinan · 2017
Later among the works it cites.
Reversible architectures for arbitrarily deep residual neural networks
Bo Chang, Lili Meng, Eldad Haber, Lars Ruthotto, David Begert, and Elliot Holtham · 2018
Later among the works it cites.
Neural ordinary differential equations
Ricky Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud · 2018
Later among the works it cites.
Compiling machine learning programs via high-level tracing, 2018
Roy Frostig, Matthew James Johnson, and Chris Leary · 2018
Later among the works it cites.
Adaptive path-integral autoencoders: Representation learning and planning for dynamical systems
Jung-Su Ha, Young-Jin Park, Hyeok-Joo Chae, Soon-Seo Park, and Han-Lim Choi · 2018
Later among the works it cites.
Learning unknown ode models with gaussian processes
Original
Markus Heinonen, Cagatay Yildiz, Henrik Mannerström, Jukka Intosalmi, and Harri Lähdesmäki · 2018
Later among the works it cites.
Mathematical Theory of Optimal Processes
Lev Semenovich Pontryagin · 2018
Later among the works it cites.
Deep neural networks motivated by partial differential equations
Original
Lars Ruthotto and Eldad Haber · 2018
Later among the works it cites.
Black-box variational inference for stochastic differential equations
Original
Thomas Ryder, Andrew Golightly, A Stephen McGough, and Dennis Prangle · 2018
Later among the works it cites.
CasADi: a software framework for nonlinear optimization and optimal control
Joel Andersson, Joris Gillis, Greg Horn, James B Rawlings, and Moritz Diehl · 2019
Later among the works it cites.
FFJORD: Free-form continuous dynamics for scalable reversible generative models
Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud · 2019
Later among the works it cites.
Deep learning with differential gaussian process flows
Pashupati Hegde, Markus Heinonen, Harri Lähdesmäki, and Samuel Kaski · 2019
Later among the works it cites.
Zygote: A differentiable programming system to bridge machine learning and scientific computing
Original
Mike Innes, Alan Edelman, Keno Fischer, Chris Rackauckus, Elliot Saba, Viral B Shah, and Will Tebbutt · 2019
Later among the works it cites.
Neural Jump Stochastic Differential Equations
Original
Junteng Jia and Austin R. Benson · 2019
Later among the works it cites.
Stochastic Flows and Jump-Diffusions
Hiroshi Kunita · 2019
Later among the works it cites.
Neural sde: Stabilizing neural ode networks with stochastic noise
Original
Xuanqing Liu, Si Si, Qin Cao, Sanjiv Kumar, and Cho-Jui Hsieh · 2019
Later among the works it cites.
Variational inference for stochastic differential equations
Manfred Opper · 2019
Later among the works it cites.
Neural stochastic differential equations
Original
Stefano Peluchetti and Stefano Favaro · 2019
Later among the works it cites.
Latent odes for irregularly-sampled time series
Yulia Rubanova, Ricky TQ Chen, and David Duvenaud · 2019
Later among the works it cites.
Applied stochastic differential equations , volume 10
Simo Särkkä and Arno Solin · 2019
Later among the works it cites.
Ode2vae: Deep generative second order odes with bayesian neural networks
Original
Çağatay Yıldız, Markus Heinonen, and Harri Lähdesmäki · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…