Fetching the paper…
Reading the bibliography…
Motivated by the resurgence of neural networks in being able to solve complex learning tasks we undertake a study of high depth networks using ReLU gates which implement the function $x \mapsto \max\{0,x\}$.
Realizations of linear functions by formulas using+
B. A. Subbotovskaya · 1961
Earlier work this paper cites.
Separating the polynomial-time hierarchy by oracles
A. C.-C. Yao · 1985
Earlier work this paper cites.
Almost optimal lower bounds for small depth circuits
J. Hastad · 1986
Earlier work this paper cites.
About one method of obtaining more than quadratic effective lower bounds of complexity of pi-schemes, 1987
A. E. Andreev · 1987
Earlier work this paper cites.
Threshold circuits of bounded depth
A. Hajnal, W. Maass, P. Pudlák, M. Szegedy, and G. Turan · 1987
Earlier work this paper cites.
Decision trees and downward closures
R. Impagliazzo and M. Naor · 1988
Earlier work this paper cites.
On small depth threshold circuits
A. A. Razborov · 1992
Earlier work this paper cites.
Shrinkage of de morgan formulae under restriction
M. S. Paterson and U. Zwick · 1993
Earlier work this paper cites.
On the computational power of depth 2 circuits with threshold and modulo gates
M. Krause and P. Pudlák · 1994
Earlier work this paper cites.
Rational approximation techniques for analysis of neural networks
K.-Y. Siu, V. P. Roychowdhury, and T. Kailath · 1994
Earlier work this paper cites.
Size–depth tradeoffs for threshold circuits
R. Impagliazzo, R. Paturi, and M. E. Saks · 1997
Earlier work this paper cites.
Bounds for the computational power and learning complexity of analog neural nets
W. Maass · 1997
Earlier work this paper cites.
The shrinkage exponent of de morgan formulas is 2
J. H. stad · 1998
Earlier work this paper cites.
Relations between communication complexity, linear arrangements, and computational complexity
J. Forster, M. Krause, S. V. Lokam, R. Mubarakzjanov, N. Schmitt, and H. U. Simon · 2001
Earlier work this paper cites.
A linear lower bound on the unbounded error probabilistic communication complexity
J. Forster · 2002
Cited alongside, same era.
On computation and communication with small bias
H. Buhrman, N. Vereshchagin, and R. de Wolf · 2007
Cited alongside, same era.
Powering requires threshold depth 3
A. A. Sherstov · 2007
Cited alongside, same era.
On the constant-depth complexity of k-clique
B. Rossman · 2008
Cited alongside, same era.
Lower bounds in communication complexity
T. Lee, A. Shraibman, et al · 2009
Cited alongside, same era.
Complexity lower bounds using linear algebra
S. V. Lokam et al · 2009
Cited alongside, same era.
Separating acˆ0 from depth-2 majority circuits
Super-linear gate and super-quadratic wire lower bounds for depth-two and depth-three threshold circuits
D. M. Kane and R. Williams · 2016
Later among the works it cites.
S. Liang and R. Srikant · 2016
Later among the works it cites.
Depth separation in relu networks for approximating smooth non-linear functions
I. Safran and O. Shamir · 2016
Later among the works it cites.
A satisfiability algorithm for depth two circuits with a sub-quadratic number of symmetric and threshold gates
S. Tamaki · 2016
Later among the works it cites.
Benefits of depth in neural networks
M. Telgarsky · 2016
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
A. A. Sherstov · 2009
Cited alongside, same era.
The sign-rank of ac ˆ0
A. A. Razborov and A. A. Sherstov · 2010
Cited alongside, same era.
The unbounded-error communication complexity of symmetric functions
A. A. Sherstov · 2011
Cited alongside, same era.
Pseudorandomness from shrinkage
R. Impagliazzo, R. Meka, and D. Zuckerman · 2012
Cited alongside, same era.
Understanding deep neural networks with rectified linear units
R. Arora, A. Basu, P. Mianjy, and A. Mukherjee · 2016
Cited alongside, same era.
Improved bounds on the sign-rank of acˆ 0
M. Bun and J. Thaler · 2016
Cited alongside, same era.
Error bounds for approximations with deep relu networks
D. Yarotsky · 2016
Later among the works it cites.
Weights at the bottom matter when the top is heavy
A. Chattopadhyay and N. S. Mande · 2017
Closest in time.
Depth separation for neural networks
A. Daniely · 2017
Closest in time.
Universal function approximation by deep neural nets with bounded width and relu activations
B. Hanin · 2017
Closest in time.
A polynomial restriction lemma with applications
V. Kabanets, D. Kane, and Z. Lu · 2017
Closest in time.
Theory ii: Landscape of the empirical risk in deep learning
T. Poggio and Q. Liao · 2017
Closest in time.
Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review
T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao · 2017
Closest in time.
Mastering the game of go without human knowledge
D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, et al · 2017
Closest in time.
Theory of deep learning iii: Generalization properties of sgd
C. Zhang, Q. Liao, A. Rakhlin, K. Sridharan, B. Miranda, N. Golowich, and T. Poggio · 2017
Closest in time.