Fetching the paper…
Reading the bibliography…
This note provides a family of classification problems, indexed by a positive integer $k$, where all shallow networks with fewer than exponentially (in $k$) many nodes exhibit error at least $1/6$, whereas a deep network with 2 nodes in each of $2k$ layers achieves zero error, as does a recurrent network with 3 distinct nodes iterated $k$ times.
On the representation of continuous functions of several variables by superpositions of continuous functions of one variable and addition
Andrey Nikolaevich Kolmogorov · 1957
Earlier work this paper cites.
Computational Limitations of Small Depth Circuits
Johan Håstad · 1986
Earlier work this paper cites.
Approximation by superpositions of a sigmoidal function
George Cybenko · 1989
Earlier work this paper cites.
Neural Network Learning: Theoretical Foundations
Martin Anthony and Peter L. Bartlett · 1999
Cited alongside, same era.
Rademacher and gaussian complexities: Risk bounds and structural results
Peter L. Bartlett and Shahar Mendelson · 2002
Cited alongside, same era.
Scaling learning algorithms towards AI
Yoshua Bengio and Yann LeCun · 2007
Cited alongside, same era.
Shallow vs. deep sum-product networks
Yoshua Bengio and Olivier Delalleau · 2011
Later among the works it cites.
On the expressive power of deep learning: A tensor analysis
Nadav Cohen, Or Sharir, and Amnon Shashua · 2015
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…