Fitting ReLUs via SGD and Quantized SGD

H. Ichimura, “Semiparametric least squares (sls) and weighted sls estimation of single- index models,” Minnesota - Center for Economic Research, Working Papers, 1991

1991

Earlier work this paper cites.

J. Horowitz and W. Härdle, “Direct semiparametric estimation of single-index models with discrete covariates,” Journal of the American Statistical Association , vol. 91, no. 436, pp. 1632–1640, 12 1996

1996

Earlier work this paper cites.

R. Collobert and J. Weston, “A unified architecture for natural language processing: Deep neural networks with multitask learning,” in Proceedings of the 25th International Conference on Machine Learning , ser. ICML ’08. New York, NY, USA: ACM, 2008, pp. 160–167

2008

Earlier work this paper cites.

T. Strohmer and R. Vershynin, “A randomized kaczmarz algorithm with exponential convergence,” Journal of Fourier Analysis and Applications , vol. 15, no. 2, p. 262, Apr 2008

2008

Earlier work this paper cites.

A. T. Kalai and R. Sastry, “The isotron algorithm: High-dimensional isotonic regression.” in COLT , 2009

2009

Earlier work this paper cites.

L. D. Dalcin, R. R. Paz, P. A. Kler, and A. Cosimo, “Parallel distributed computing using python,” Advances in Water Resources , vol. 34, no. 9, pp. 1124–1139, 2011

2011

Earlier work this paper cites.

E. Moulines and F. R. Bach, “Non-asymptotic analysis of stochastic approximation algorithms for machine learning,” in Advances in Neural Information Processing Systems 24 , J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2011, pp. 451–459

2011

Earlier work this paper cites.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 , ser. NIPS’12. USA: Curran Associates Inc., 2012, pp. 1097–1105

2012

Earlier work this paper cites.

J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, M. aurelio Ranzato, A. Senior, P. Tucker, K. Yang, Q. V. Le, and A. Y. Ng, “Large scale distributed deep networks,” in Advances in Neural Information Processing Systems 25 , F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2012, pp. 1223–1231

2012

Earlier work this paper cites.

D. Needell, R. Ward, and N. Srebro, “Stochastic gradient descent, weighted sampling, and the randomized kaczmarz algorithm,” in Advances in Neural Information Processing Systems 27 , Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2014, pp. 1017–1025

2014

Earlier work this paper cites.

F. Seide, H. Fu, J. Droppo, G. Li, and D. Yu, “1-bit stochastic gradient descent and application to data-parallel distributed training of speech dnns,” in Interspeech 2014 , September 2014

2014

Cited alongside, same era.

C. M. De Sa, C. Zhang, K. Olukotun, C. Ré, and C. Ré, “Taming the wild: A unified analysis of hogwild-style algorithms,” in Advances in Neural Information Processing Systems 28 , C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, Eds. Curran Associates, Inc., 2015, pp. 2674–2682

2015

Cited alongside, same era.

S. Goel, V. Kanade, A. R. Klivans, and J. Thaler, “Reliably learning the relu in polynomial time,” CoRR , vol. abs/1611.10258, 2016

Original

2016

Cited alongside, same era.

D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic, “Qsgd: Communication-efficient sgd via gradient quantization and encoding,” in Advances in Neural Information Processing Systems 30 , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds. Curran Associates, Inc., 2017, pp. 1709–1720

G. Jagatap and C. Hegde, “Learning relu networks via alternating minimization,” arXiv preprint arXiv:1806.07863 , 2018

Original

2018

Later among the works it cites.

X. Zhang, Y. Yu, L. Wang, and Q. Gu, “Learning one-hidden-layer relu networks via gradient descent,” arXiv preprint arXiv:1806.07808 , 2018

Original

2018

Later among the works it cites.

S. Liang, R. Sun, Y. Li, and R. Srikant, “Understanding the loss surface of single-layered neural networks for binary classification,” CoRR , vol. abs/1803.00909, 2018

Original

2018

Later among the works it cites.

S. Goel, A. Klivans, and R. Meka, “Learning one convolutional layer with overlapping patches,” arXiv preprint arXiv:1802.02547 , 2018

Original

2018

Later among the works it cites.

H. Fu, Y. Chi, and Y. Liang, “Local geometry of one-hidden-layer neural networks for logistic regression,” arXiv preprint arXiv:1802.06463 , 2018