Fetching the paper…
Reading the bibliography…
Motivated by the gap between theoretical optimal approximation rates of deep neural networks (DNNs) and the accuracy realized in practice, we seek to improve the training of DNNs.
The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate
G. H. Golub and V. Pereyra · 1973
Earlier work this paper cites.
Approximation by superpositions of a sigmoidal function
George Cybenko · 1989
Earlier work this paper cites.
Optimal nonlinear approximation
Ronald A DeVore, Ralph Howard, and Charles Micchelli · 1989
Earlier work this paper cites.
Nonlinear Approximation
Ronald A DeVore · 1998
Earlier work this paper cites.
Separable nonlinear least squares: the variable projection method and its applications
Gene Golub and Victor Pereyra · 2003
Earlier work this paper cites.
Variable projections neural network training
V. Pereyra, G. Scherer, and F. Wong · 2004
Earlier work this paper cites.
Projecting to a slow manifold: Singularly perturbed systems and legacy codes
C William Gear, Tasso J Kaper, Ioannis G Kevrekidis, and Antonios Zagaris · 2005
Earlier work this paper cites.
Numerical Optimization
Jorge Nocedal and Stephen Wright · 2006
Earlier work this paper cites.
A Course in Approximation Theory , volume 101
Elliott Ward Cheney and William Allan Light · 2009
Earlier work this paper cites.
Compressed sensing and best k k -term approximation
Albert Cohen, Wolfgang Dahmen, and Ronald DeVore · 2009
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio · 2010
Earlier work this paper cites.
Machine Learning: A Probabilistic Perspective
Kevin P Murphy · 2012
Cited alongside, same era.
Adam: A method for stochastic optimization, 2014
Diederik P. Kingma and Jimmy Ba · 2014
Cited alongside, same era.
TensorFlow: Large-scale machine learning on heterogeneous systems, 2015
Martín Abadi et al · 2015
Cited alongside, same era.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Cited alongside, same era.
Representation benefits of deep feedforward networks
Matus Telgarsky · 2015
Cited alongside, same era.
Understanding deep neural networks with rectified linear units
Error bounds for approximations with deep ReLU networks
Dmitry Yarotsky · 2017
Later among the works it cites.
Neural ordinary differential equations
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud · 2018
Later among the works it cites.
How to start training: The effect of initialization and architecture
Boris Hanin and David Rolnick · 2018
Later among the works it cites.
ReLU deep neural networks and linear finite elements
Juncai He, Lin Li, Jinchao Xu, and Chunyue Zheng · 2018
Later among the works it cites.
Collapse of deep and narrow neural nets
Lu Lu, Yanhui Su, and George Em Karniadakis · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee · 2016
Cited alongside, same era.
Certified reduced basis methods for parametrized partial differential equations
Jan S Hesthaven, Gianluigi Rozza, Benjamin Stamm, et al · 2016
Cited alongside, same era.
Stable architectures for deep neural networks
Eldad Haber and Lars Ruthotto · 2017
Cited alongside, same era.
Universal function approximation by deep neural nets with bounded width and ReLU activations
Boris Hanin · 2017
Cited alongside, same era.
Approximating continuous functions by ReLU nets of minimal width
Boris Hanin and Mark Sellke · 2017
Cited alongside, same era.
Automatic differentiation in PyTorch
Adam Paszke et al · 2017
Cited alongside, same era.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun
Cited in the paper.
Workshop Report on Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial Intelligence
Nathan Baker, Frank Alexander, Timo Bremer, Aric Hagberg, Yannis Kevrekidis, Habib Najm, Manish Parashar, Abani Patra, James Sethian, Stefan Wild, et al · 2019
Closest in time.
Growing axons: greedy learning of neural networks with application to function approximation
Daria Fokina and Ivan Oseledets · 2019
Closest in time.
Dying relu and initialization: Theory and numerical examples
Lu Lu, Yeonjong Shin, Yanhui Su, and George Em Karniadakis · 2019
Closest in time.
Deep ReLU networks and high-order finite element methods
Joost AA Opschoor, Philipp Petersen, and Christoph Schwab · 2019
Closest in time.
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
Maziar Raissi, Paris Perdikaris, and George E Karniadakis · 2019
Closest in time.
Stochastic conditional generative networks with basis decomposition
Ze Wang, Xiuyuan Cheng, Guillermo Sapiro, and Qiang Qiu · 2019
Closest in time.