$\mathcal{G}$-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space

Learning representations by back-propagating errors

David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams · 1986

Earlier work this paper cites.

Linear Algebra and its applications, 1997

David C Lay · 1997

Earlier work this paper cites.

Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit

Richard HR Hahnloser, Rahul Sarpeshkar, Misha A Mahowald, Rodney J Douglas, and H Sebastian Seung · 2000

Earlier work this paper cites.

Learning multiple layers of features from tiny images

Alex Krizhevsky and Geoffrey Hinton · 2009

Earlier work this paper cites.

Symmetry-invariant optimization in deep networks

Original

Vijay Badrinarayanan, Bamdev Mishra, and Roberto Cipolla · 2015

Earlier work this paper cites.

Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015

Earlier work this paper cites.

Deep learning

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton · 2015

Cited alongside, same era.

Path-sgd: Path-normalized optimization in deep neural networks

Behnam Neyshabur, Ruslan R Salakhutdinov, and Nati Srebro · 2015

Cited alongside, same era.

Wide & deep learning for recommender systems

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al · 2016

Cited alongside, same era.

Wide residual networks

Original

Sergey Zagoruyko and Nikos Komodakis · 2016

Cited alongside, same era.

Densely connected convolutional networks

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger · 2017

Cited alongside, same era.

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017

Later among the works it cites.

Deep & cross network for ad click predictions

Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang · 2017

Later among the works it cites.

Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms

Han Xiao, Kashif Rasul, and Roland Vollgraf · 2017

Later among the works it cites.

Algorithmic regularization in learning deep homogeneous models: Layers are automatically balanced

Simon S Du, Wei Hu, and Jason D Lee · 2018

Closest in time.

Capacity control of relu neural networks by basis-path norm

Original

Shuxin Zheng, Qi Meng, Huishuai Zhang, Wei Chen, Nenghai Yu, and Tie-Yan Liu · 2018

Closest in time.