Fetching the paper…
Reading the bibliography…
We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the orthogonal group O(d).
Exponential operators and parameter differentiation in quantum physics
R. M. Wilcox · 1967
Earlier work this paper cites.
Foundations of differentiable manifolds and Lie groups
F. W. Warner · 1971
Earlier work this paper cites.
Representations of compact lie groups
T. Bröcker and T. tom Dieck · 1985
Earlier work this paper cites.
Isospectral flows and abstract matrix factorizations
M. T. Chu and L. K. Norris · 1988
Earlier work this paper cites.
Least squares matching problems
R. W. Brockett · 1989
Earlier work this paper cites.
Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems
R. W. Brockett · 1991
Earlier work this paper cites.
The problem of learning long-term dependencies in recurrent networks
Y. Bengio, P. Frasconi, and P. Y. Simard · 1993
Earlier work this paper cites.
The geometry of algorithms with orthogonality constraints
A. Edelman, T. A. Arias, and S. T. Smith · 1998
Earlier work this paper cites.
Applications of Lie Groups to Differential Equations
P. J. Olver · 2000
Earlier work this paper cites.
An introduction to numerical analysis
E. Süli and D. F. Mayers · 2003
Earlier work this paper cites.
An introduction to differential geometry with applications to elasticity
P. G. Ciarlet · 2005
Earlier work this paper cites.
Important aspects of geometric numerical integration
E. Hairer · 2005
Earlier work this paper cites.
A dynamical systems approach to weighted graph matching
M. M. Zavlanos and G. J. Pappas · 2006
Earlier work this paper cites.
Optimization Algorithms on Matrix Manifolds
P. Absil, R. E. Mahony, and R. Sepulchre · 2008
Earlier work this paper cites.
Introduction to smooth manifolds. 2nd revised ed
J. Lee · 2012
Earlier work this paper cites.
Stochastic gradient descent on riemannian manifolds
S. Bonnabel · 2013
Earlier work this paper cites.
On the difficulty of training recurrent neural networks
R. Pascanu, T. Mikolov, and Y. Bengio · 2013
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
A. M. Saxe, J. L. McClelland, and S. Ganguli · 2014
Earlier work this paper cites.
Coordinate-descent for learning orthogonal matrices through Givens rotations
U. Shalit and G. Chechik · 2014
Earlier work this paper cites.
Unitary evolution recurrent neural networks
M. Arjovsky, A. Shah, and Y. Bengio · 2016
Cited alongside, same era.
Deep Learning
I. J. Goodfellow, Y. Bengio, and A. C. Courville · 2016
Cited alongside, same era.
D. Ha, A. M. Dai, and Q. V. Le · 2016
Cited alongside, same era.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Cited alongside, same era.
Recurrent orthogonal networks and long-memory tasks
M. Henaff, A. Szlam, and Y. LeCun · 2016
Cited alongside, same era.
Stable architectures for deep neural networks
E. Haber and L. Ruthotto · 2017
Cited alongside, same era.
Differentiable ranking and sorting using optimal transport
M. Cuturi, O. Teboul, and J. Vert · 2019
Later among the works it cites.
Augmented neural odes
E. Dupont, A. Doucet, and Y. W. Teh · 2019
Later among the works it cites.
Stochastic optimization of sorting networks via continuous relaxations
A. Grover, E. Wang, A. Zweig, and S. Ermon · 2019
Later among the works it cites.
B. Güler, A. Laignelet, and P. Parpas · 2019
Later among the works it cites.
Orthogonal deep neural networks
K. Jia, S. Li, Y. Wen, T. Liu, and D. Tao · 2019
Later among the works it cites.
Neural SDE: stabilizing neural ODE networks with stochastic noise
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Tunable efficient unitary neural networks (EUNN) and their application to rnns
L. Jing, Y. Shen, T. Dubcek, J. Peurifoy, S. A. Skirlo, Y. LeCun, M. Tegmark, and M. Soljacic · 2017
Cited alongside, same era.
Efficient orthogonal parametrisation of recurrent neural networks using Householder reflections
Z. Mhammedi, A. D. Hellicar, A. Rahman, and J. Bailey · 2017
Cited alongside, same era.
Evolution strategies as a scalable alternative to reinforcement learning
T. Salimans, J. Ho, X. Chen, and I. Sutskever · 2017
Cited alongside, same era.
All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation
D. Xie, J. Xiong, and S. Pu · 2017
Cited alongside, same era.
Can we gain more from orthogonality regularizations in training deep networks?
N. Bansal, X. Chen, and Z. Wang · 2018
Cited alongside, same era.
Reversible architectures for arbitrarily deep residual neural networks
B. Chang, L. Meng, E. Haber, L. Ruthotto, D. Begert, and E. Holtham · 2018
Cited alongside, same era.
X. Liu, T. Xiao, S. Si, Q. Cao, S. Kumar, and C. Hsieh · 2019
Later among the works it cites.
Mnist-c: A robustness benchmark for computer vision
N. Mu and J. Gilmer · 2019
Later among the works it cites.
Latent ordinary differential equations for irregularly-sampled time series
Y. Rubanova, T. Q. Chen, and D. Duvenaud · 2019
Later among the works it cites.
Orthogonal convolutional neural networks
J. Wang, Y. Chen, R. Chakraborty, and S. X. Yu · 2019
Later among the works it cites.
Benchmarking model-based reinforcement learning
T. Wang, X. Bao, I. Clavera, J. Hoang, Y. Wen, E. Langlois, S. Zhang, G. Zhang, P. Abbeel, and J. Ba · 2019
Later among the works it cites.
ANODEV2: A coupled neural ODE framework
T. Zhang, Z. Yao, A. Gholami, J. E. Gonzalez, K. Keutzer, M. W. Mahoney, and G. Biros · 2019
Later among the works it cites.
Fast differentiable sorting and ranking
M. Blondel, O. Teboul, Q. Berthet, and J. Djolonga · 2020
Closest in time.
Stochastic flows and geometric optimization on the orthogonal group
K. Choromanski, D. Cheikhi, J. Davis, V. Likhosherstov, A. Nazaret, A. Bahamou, X. Song, M. Akarte, J. Parker-Holder, J. Bergquist, Y. Gao, A. Pacchiano, T. Sarlós, A. Weller, and V. Sindhwani · 2020
Closest in time.
Time dependence in non-autonomous neural odes
J. Q. Davis, K. Choromanski, J. Varley, H. Lee, J. E. Slotine, V. Likhosterov, A. Weller, A. Makadia, and V. Sindhwani · 2020
Closest in time.
C. Finlay, J. Jacobsen, L. Nurbekyan, and A. M. Oberman · 2020
Closest in time.
Provable benefit of orthogonal initialization in optimizing deep linear networks
W. Hu, L. Xiao, and J. Pennington · 2020
Closest in time.
CWY parametrization for scalable learning of orthogonal and stiefel matrices
V. Likhosherstov, J. Davis, K. Choromanski, and A. Weller · 2020
Closest in time.
S. Massaroli, M. Poli, M. Bin, J. Park, A. Yamashita, and H. Asama · 2020
Closest in time.