Fetching the paper…
Reading the bibliography…
Existing depth separation results for constant-depth networks essentially show that certain radial functions in $\mathbb{R}^d$, which can be easily approximated with depth $3$ networks, cannot be approximated by depth $2$ networks, even up to constant accuracy, unless their size is exponential in $d$.
Inverses of vandermonde matrices
N. Macon and A. Spitzbart · 1958
Earlier work this paper cites.
Universal approximation bounds for superpositions of a sigmoidal function
A. R. Barron · 1993
Earlier work this paper cites.
Special functions, volume 71 of encyclopedia of mathematics and its applications, 1999
G. E. Andrews, R. Askey, and R. Roy · 1999
Earlier work this paper cites.
Handbook of beta distribution and its applications
A. K. Gupta and S. Nadarajah · 2004
Earlier work this paper cites.
A note on approximation of a ball by polytopes
M. Kochol · 2004
Earlier work this paper cites.
Theory of classification: A survey of some recent advances
S. Boucheron, O. Bousquet, and G. Lugosi · 2005
Earlier work this paper cites.
Theory of probability and random processes
L. Koralov and Y. G. Sinai · 2007
Earlier work this paper cites.
Distributing points on the sphere: partitions, separation, quadrature and energy
P. Leopardi · 2007
Earlier work this paper cites.
Agnostically learning halfspaces
A. T. Kalai, A. R. Klivans, Y. Mansour, and R. A. Servedio · 2008
Earlier work this paper cites.
Shallow vs. deep sum-product networks
O. Delalleau and Y. Bengio · 2011
Earlier work this paper cites.
Moments and absolute moments of the normal distribution
A. Winkelbauer · 2012
Earlier work this paper cites.
On the representational efficiency of restricted boltzmann machines
J. Martens, A. Chattopadhya, T. Pitassi, and R. Zemel · 2013
Cited alongside, same era.
Embedding hard learning problems into gaussian space
A. R. Klivans and P. Kothari · 2014
Cited alongside, same era.
On the expressive efficiency of sum product networks
J. Martens and V. Medabalimi · 2014
Cited alongside, same era.
On the number of linear regions of deep neural networks
G. F. Montufar, R. Pascanu, K. Cho, and Y. Bengio · 2014
Cited alongside, same era.
Understanding machine learning: From theory to algorithms
S. Shalev-Shwartz and S. Ben-David · 2014
Cited alongside, same era.
Provable approximation properties for deep neural networks
U. Shaham, A. Cloninger, and R. R. Coifman · 2016
Later among the works it cites.
Benefits of depth in neural networks
M. Telgarsky · 2016
Later among the works it cites.
Error bounds for approximations with deep relu networks
D. Yarotsky · 2016
Later among the works it cites.
Depth separation for neural networks
A. Daniely · 2017
Later among the works it cites.
Depth-width tradeoffs in approximating natural functions with neural networks
I. Safran and O. Shamir · 2017
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
N. Cohen, O. Sharir, and A. Shashua · 2015
Cited alongside, same era.
The power of depth for feedforward neural networks
R. Eldan and O. Shamir · 2016
Cited alongside, same era.
S. Liang and R. Srikant · 2016
Cited alongside, same era.
Why and when can deep–but not shallow–networks avoid the curse of dimensionality: a review
T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao · 2016
Cited alongside, same era.
Exponential expressivity in deep neural networks through transient chaos
B. Poole, S. Lahiri, M. Raghu, J. Sohl-Dickstein, and S. Ganguli · 2016
Cited alongside, same era.
S. Shalev-Shwartz, O. Shamir, and S. Shammah · 2017
Later among the works it cites.
On the complexity of learning neural networks
L. Song, S. Vempala, J. Wilmes, and B. Xie · 2017
Later among the works it cites.
Provable limitations of deep learning
E. Abbe and C. Sandon · 2018
Later among the works it cites.
Approximation by combinations of relu and squared relu ridge functions with l1 and l0 controls
J. M. Klusowski and A. R. Barron · 2018
Later among the works it cites.
Distribution-specific hardness of learning neural networks
O. Shamir · 2018
Later among the works it cites.