Fetching the paper…
Reading the bibliography…
We show that there is a simple (approximately radial) function on $\reals^d$, expressible by a small 3-layer feedforward neural networks, which cannot be approximated by any 2-layer network, to more than a certain constant accuracy, unless its width is exponential in the dimension.
Almost optimal lower bounds for small depth circuits
J. Håstad · 1986
Earlier work this paper cites.
Learning internal representations by error propagation
D. E. Rumelhart, G. E. Hinton, and R. J. Williams · 1986
Earlier work this paper cites.
Approximation by superpositions of a sigmoidal function
G. Cybenko · 1989
Earlier work this paper cites.
On the approximate realization of continuous mappings by neural networks
Ken-Ichi Funahashi · 1989
Earlier work this paper cites.
Multilayer feedforward networks are universal approximators
K. Hornik, M. Stinchcombe, and H. White · 1989
Earlier work this paper cites.
Degree of approximation by superpositions of a sigmoidal function
C. Debao · 1993
Earlier work this paper cites.
Threshold circuits of bounded depth
András Hajnal, Wolfgang Maass, Pavel Pudlák, Márió Szegedy, and György Turán · 1993
Earlier work this paper cites.
Approximation and estimation bounds for artificial neural networks
Andrew R. Barron · 1994
Earlier work this paper cites.
A comparison of the computational power of sigmoid and boolean threshold circuits
W. Maass, G. Schnitger, and E. Sontag · 1994
Earlier work this paper cites.
Circuit complexity and neural networks
I. Parberry · 1994
Cited alongside, same era.
Applied analysis
John K. Hunter and Bruno Nachtergaele · 2001
Cited alongside, same era.
Tail-Sensitive Gaussian Asymptotics for Marginals of Concentrated Measures in High Dimension
Sasha Sodin · 2007
Cited alongside, same era.
Arithmetic circuits: A survey of recent results and open questions
A. Shpilka and A. Yehudayoff · 2010
Cited alongside, same era.
Shallow vs. deep sum-product networks
O. Delalleau and Y. Bengio · 2011
Cited alongside, same era.
On fourier transforms of radial functions and distributions
Loukas Grafakos and Gerald Teschl · 2013
Cited alongside, same era.
Approximations for the bessel and airy functions with an explicit error term
Ilia Krasikov · 2014
Later among the works it cites.
On the expressive efficiency of sum product networks
J. Martens and V. Medabalimi · 2014
Later among the works it cites.
On the number of linear regions of deep neural networks
G. F Montufar, R. Pascanu, K. Cho, and Y. Bengio · 2014
Later among the works it cites.
On the expressive power of deep learning: A tensor analysis
N. Cohen, O. Sharir, and A. Shashua · 2015
Closest in time.
NIST Digital Library of Mathematical Functions
DLMF · 2015
Closest in time.
Private Communication, 2015
James Martens · 2015
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
James Martens, Arkadev Chattopadhya, Toni Pitassi, and Richard Zemel · 2013
Cited alongside, same era.
On the number of inference regions of deep feed forward networks with piece-wise linear activations≫
R. Pascanu, G. Montufar, and Y. Bengio · 2013
Cited alongside, same era.
On the complexity of shallow and deep neural network classifiers
M. Bianchini and F. Scarselli · 2014
Cited alongside, same era.
Closest in time.
An average-case depth hierarchy theorem for boolean circuits
B. Rossman, R. Servedio, and L.-Y. Tan · 2015
Closest in time.
Representation benefits of deep feedforward networks
M. Telgarsky · 2015
Closest in time.