Built on

H. Robertson, “The solution of a set of reaction rate equations,” Numerical analysis: an introduction 178182
1966
Earlier work this paper cites.
L. F. Shampine and C. W. Gear, “A user’s view of solving stiff ordinary differential equations,” SIAM review 21
1979
Earlier work this paper cites.
L. Petzold, “Differential/algebraic equations are not ode’s,” SIAM Journal on Scientific and Statistical Computing 3
Original
1982
Earlier work this paper cites.
J. G. Verwer, “Gauss–seidel iteration for stiff odes from chemical kinetics,” SIAM Journal on Scientific Computing 15
1994
Earlier work this paper cites.
G. Wanner and E. Hairer, Solving ordinary differential equations II , Vol. 375 (Springer Berlin Heidelberg, 1996)
1996
Earlier work this paper cites.
C. Rackauckas, Y. Ma, J. Martensen, C. Warner, K. Zubov, R. Supekar, D. Skinner, A. Ramadhan, and A. Edelman, “Universal differential equations for scientific machine learning,” arXiv preprint arXiv:2001.04385 (2020a)
Original
2001
Earlier work this paper cites.
R. Serban and A. C. Hindmarsh, “Cvodes: An ode solver with sensitivity analysis capabilities,” Tech. Rep. (Technical Report UCRL-JP-200039, Lawrence Livermore National Laboratory, 2003)
2003
Earlier work this paper cites.
J. Liesen and P. Tichỳ, “Convergence analysis of krylov subspace methods,” GAMM-Mitteilungen 27
2004
Earlier work this paper cites.
A. C. Hindmarsh, P. N. Brown, K. E. Grant, S. L. Lee, R. Serban, D. E. Shumaker, and C. S. Woodward, “SUNDIALS: Suite of nonlinear and differential/algebraic equation solvers,” ACM Transactions on Mathematical Software (TOMS) 31
2005
Earlier work this paper cites.
L. F. Shampine and S. Thompson, “Stiff systems,” Scholarpedia 2
2007
Earlier work this paper cites.
R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training recurrent neural networks,” in International conference on machine learning (PMLR, 2013) pp. 1310–1318
2013
Earlier work this paper cites.
H. Zhang and A. Sandu, “Fatode: a library for forward, adjoint, and tangent linear integration of odes,” SIAM Journal on Scientific Computing 36
2014
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014)
Original
2014
Earlier work this paper cites.
L. F. Shampine and L. O. Jay, “Dense output,” Encyclopedia of Applied and Computational Mathematics; Springer: Berlin, Germany , 339–345 (2015)
2015
Earlier work this paper cites.
Note many methods do not need to store all stages. Additionally, some higher order Runge-Kutta methods require a few additional f f evaluations in order to generate an interpolation matching the order of the solver and thus can default to interpolants of an order less or more. For a full discussion of dense output, see Shampine and Jay 2015
2015
Earlier work this paper cites.

Similar

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International conference on machine learning (PMLR, 2015) pp. 448–456
2015
Cited alongside, same era.
S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems,” Proceedings of the national academy of sciences 113
2016
Cited alongside, same era.
N. M. Mangan, S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Inferring biological networks by sparse identification of nonlinear dynamics,” IEEE Transactions on Molecular, Biological and Multi-Scale Communications 2
2016
Cited alongside, same era.
J. Revels, M. Lubin, and T. Papamarkou, “Forward-mode automatic differentiation in julia,” arXiv preprint arXiv:1607.07892 (2016)
Original
2016
Cited alongside, same era.

Then

A. Gholami, K. Keutzer, and G. Biros, “Anode: Unconditionally accurate memory-efficient gradients for neural odes,” arXiv preprint arXiv:1902.10298 (2019)
Original
2019
Later among the works it cites.
S. Bai, J. Z. Kolter, and V. Koltun, “Deep equilibrium models,” arXiv preprint arXiv:1909.01377 (2019)
Original
2019
Later among the works it cites.
B. Amos, Differentiable optimization-based modeling for machine learning , Ph.D. thesis, PhD thesis. Carnegie Mellon University (2019)
2019
Later among the works it cites.
W. Ji and S. Deng, “Autonomous discovery of unknown reaction pathways from data by chemical reaction neural network,” arXiv preprint arXiv:2002.09062 (2020)
Original
2020
Later among the works it cites.
A. Bills, S. Sripad, W. L. Fredericks, M. Guttenberg, D. Charles, E. Frank, and V. Viswanathan, “Universal battery performance and degradation model for electric aircraft,” arXiv preprint arXiv:2008.01527 (2020)

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2021

Stiff Neural Ordinary Differential Equations

Kim, Suyong, Ji, Weiqi, Deng, Sili et al.

Understand

Neural Ordinary Differential Equations (ODE) are a promising approach to learn dynamic models from time-series data in science and engineering applications.

This work aims at learning Neural ODE for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems.
We first show the challenges of learning neural ODE in the classical stiff ODE systems of Robertson's problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems.
We then present successful demonstrations in stiff systems of Robertson's problem and an air pollution problem.

Built on

H. Robertson, “The solution of a set of reaction rate equations,” Numerical analysis: an introduction 178182
1966
Earlier work this paper cites.
L. F. Shampine and C. W. Gear, “A user’s view of solving stiff ordinary differential equations,” SIAM review 21
1979
Earlier work this paper cites.
L. Petzold, “Differential/algebraic equations are not ode’s,” SIAM Journal on Scientific and Statistical Computing 3
Original
1982
Earlier work this paper cites.
J. G. Verwer, “Gauss–seidel iteration for stiff odes from chemical kinetics,” SIAM Journal on Scientific Computing 15
1994
Earlier work this paper cites.
G. Wanner and E. Hairer, Solving ordinary differential equations II , Vol. 375 (Springer Berlin Heidelberg, 1996)
1996
Earlier work this paper cites.
C. Rackauckas, Y. Ma, J. Martensen, C. Warner, K. Zubov, R. Supekar, D. Skinner, A. Ramadhan, and A. Edelman, “Universal differential equations for scientific machine learning,” arXiv preprint arXiv:2001.04385 (2020a)
Original
2001
Earlier work this paper cites.
R. Serban and A. C. Hindmarsh, “Cvodes: An ode solver with sensitivity analysis capabilities,” Tech. Rep. (Technical Report UCRL-JP-200039, Lawrence Livermore National Laboratory, 2003)
2003
Earlier work this paper cites.
J. Liesen and P. Tichỳ, “Convergence analysis of krylov subspace methods,” GAMM-Mitteilungen 27
2004
Earlier work this paper cites.
A. C. Hindmarsh, P. N. Brown, K. E. Grant, S. L. Lee, R. Serban, D. E. Shumaker, and C. S. Woodward, “SUNDIALS: Suite of nonlinear and differential/algebraic equation solvers,” ACM Transactions on Mathematical Software (TOMS) 31
2005
Earlier work this paper cites.
L. F. Shampine and S. Thompson, “Stiff systems,” Scholarpedia 2
2007
Earlier work this paper cites.
R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training recurrent neural networks,” in International conference on machine learning (PMLR, 2013) pp. 1310–1318
2013
Earlier work this paper cites.
H. Zhang and A. Sandu, “Fatode: a library for forward, adjoint, and tangent linear integration of odes,” SIAM Journal on Scientific Computing 36
2014
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014)
Original
2014
Earlier work this paper cites.
L. F. Shampine and L. O. Jay, “Dense output,” Encyclopedia of Applied and Computational Mathematics; Springer: Berlin, Germany , 339–345 (2015)
2015
Earlier work this paper cites.
Note many methods do not need to store all stages. Additionally, some higher order Runge-Kutta methods require a few additional f f evaluations in order to generate an interpolation matching the order of the solver and thus can default to interpolants of an order less or more. For a full discussion of dense output, see Shampine and Jay 2015
2015
Earlier work this paper cites.

Similar

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International conference on machine learning (PMLR, 2015) pp. 448–456
2015
Cited alongside, same era.
S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems,” Proceedings of the national academy of sciences 113
2016
Cited alongside, same era.
N. M. Mangan, S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Inferring biological networks by sparse identification of nonlinear dynamics,” IEEE Transactions on Molecular, Biological and Multi-Scale Communications 2
2016
Cited alongside, same era.
J. Revels, M. Lubin, and T. Papamarkou, “Forward-mode automatic differentiation in julia,” arXiv preprint arXiv:1607.07892 (2016)
Original
2016
Cited alongside, same era.

Then

A. Gholami, K. Keutzer, and G. Biros, “Anode: Unconditionally accurate memory-efficient gradients for neural odes,” arXiv preprint arXiv:1902.10298 (2019)
Original
2019
Later among the works it cites.
S. Bai, J. Z. Kolter, and V. Koltun, “Deep equilibrium models,” arXiv preprint arXiv:1909.01377 (2019)
Original
2019
Later among the works it cites.
B. Amos, Differentiable optimization-based modeling for machine learning , Ph.D. thesis, PhD thesis. Carnegie Mellon University (2019)
2019
Later among the works it cites.
W. Ji and S. Deng, “Autonomous discovery of unknown reaction pathways from data by chemical reaction neural network,” arXiv preprint arXiv:2002.09062 (2020)
Original
2020
Later among the works it cites.
A. Bills, S. Sripad, W. L. Fredericks, M. Guttenberg, D. Charles, E. Frank, and V. Viswanathan, “Universal battery performance and degradation model for electric aircraft,” arXiv preprint arXiv:2008.01527 (2020)

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” arXiv preprint arXiv:1606.08415 (2016)

Original

2016

Cited alongside, same era.

J. Bezanson, A. Edelman, S. Karpinski, and V. B. Shah, “Julia: A fresh approach to numerical computing,” SIAM Review 59

2017

Cited alongside, same era.

C. Rackauckas and Q. Nie, “Differentialequations. jl–a performant and feature-rich ecosystem for solving differential equations in julia,” Journal of Open Research Software 5

2017

Cited alongside, same era.

B. Amos and J. Z. Kolter, “Optnet: Differentiable optimization as a layer in neural networks,” in International Conference on Machine Learning (PMLR, 2017) pp. 136–145

2017

Cited alongside, same era.

R. T. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud, “Neural ordinary differential equations,” arXiv preprint arXiv:1806.07366 (2018)

Original

2018

Cited alongside, same era.

M. Innes, “Flux: Elegant machine learning with julia,” Journal of Open Source Software 3

2018

Cited alongside, same era.

Y. Rubanova, R. T. Chen, and D. Duvenaud, “Latent ordinary differential equations for irregularly-sampled time series,” 33rd Conference on Neural Information Processing Systems (2019)

2019

Cited alongside, same era.

C. Rackauckas, M. Innes, Y. Ma, J. Bettencourt, L. White, and V. Dixit, “Diffeqflux. jl-a julia library for neural differential equations,” arXiv preprint arXiv:1902.02376 (2019)

Original

2019

Cited alongside, same era.

Original

2020