Fetching the paper…
Reading the bibliography…
It is well-known that the training of Deep Neural Networks (DNN) can be formalized in the language of optimal control.
“Linear Programming and Economic Analysis”
R. Dorfman, P.A. Samuelson and R.M. Solow · 1958
Earlier work this paper cites.
“A Steepest-Ascent Method for Solving Optimum Programming Problems”
A.E. Bryson and W.F. Denham · 1962
Earlier work this paper cites.
“Dissipative dynamical systems part I: General theory”
J.C. Willems · 1972
Earlier work this paper cites.
“Turnpike theory”
L.W. McKenzie · 1976
Earlier work this paper cites.
“Learning to tell two spirals apart”
Kevin Lang and Michael Witbrock · 1988
Earlier work this paper cites.
“Approximation by superpositions of a sigmoidal function”
George Cybenko · 1989
Earlier work this paper cites.
“Approximation capabilities of multilayer feedforward networks”
Kurt Hornik · 1991
Earlier work this paper cites.
“Infinite Horizon Optimal Control: Deterministic and Stochastic Systems”
D.A. Carlson, A. Haurie and A. Leizarowitz · 1991
Earlier work this paper cites.
“Complete controllability of continuous-time recurrent neural networks”
Eduardo Sontag and H“’ector Sussmann · 1997
Earlier work this paper cites.
“Complete Controllability of Discrete-Time Recurrent Neural Networks”, 2000
Thomas Steinberger and Lucas Zinner · 2000
Earlier work this paper cites.
“Constrained formulations for neural network training and their applications to solve the two-spiral problem”
B.. Wah and M. Qian · 2000
Cited alongside, same era.
“Variations of the two-spiral task”
Stephan. Chalup and Lukasz Wiklendt · 2007
Cited alongside, same era.
“On Average Performance and Stability of Economic Model Predictive Control”
D. Angeli, R. Amrit and J.B. Rawlings · 2012
Cited alongside, same era.
“Practical Methods of Optimization”
Roger Fletcher · 2013
Cited alongside, same era.
“Do Deep Nets Really Need to be Deep?”
Jimmy Ba and Rich Caruana · 2014
Cited alongside, same era.
“Understanding Machine Learning: From Theory to Algorithms”
Shai Shalev-Shwartz and Shai Ben-David · 2014
Cited alongside, same era.
“Optimal Neumann control for the 1D wave equation: Finite horizon, infinite horizon, boundary tracking terms and the turnpike property”
M. Gugat, E. Tr“’elat and E. Zuazua · 2016
Later among the works it cites.
“Automatic differentiation in machine learning: a survey”
Atlm“”unes Baydin, Barak Pearlmutter, Alexey Radul and Jeffrey Siskind · 2017
Later among the works it cites.
“Maximum principle based algorithms for deep learning”
Qianxiao Li, Long Chen, Cheng Tai and E Weinan · 2017
Later among the works it cites.
“On Turnpike and Dissipativity Properties of Continuous-Time Optimal Control Problems”
T. Faulwasser, M. Korda, C.N. Jones and D. Bonvin · 2017
Later among the works it cites.
“Economic Nonlinear Model Predictive Control: Stability, Optimality and Performance”
T. Faulwasser, L. Gr“”une and M. M“”uller · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
“Gradient-based Hyperparameter Optimization through Reversible Learning”
Dougal Maclaurin, David Duvenaud and Ryan Adams · 2015
Cited alongside, same era.
“Deep Learning”
Ian Goodfellow, Yoshua Bengio and Aaron Courville · 2016
Cited alongside, same era.
“On the relation between strict dissipativity and turnpike properties”
L. Gr“”une and M.A. M“”uller · 2016
Cited alongside, same era.
J.A.E. Andersson, J. Gillis, G. Horn, J.B. Rawlings and M. Diehl · 2019
Later among the works it cites.
“Large-time asymptotics in deep learning”
Carlos Esteve, Borjan Geshkovski, Dario Pighin and Enrique Zuazua · 2020
Later among the works it cites.
“Turnpike Properties in Discrete-Time Mixed Integer Optimal Control” arxiv: 2002.02049
T. Faulwasser and A. Murray · 2020
Later among the works it cites.
“Turnpike Properties in Optimal Control: An Overview of Discrete-Time and Continuous-Time Results” Submitted. arxiv: 2011.13670
T. Faulwasser and L. Gr“”une · 2020
Later among the works it cites.