A New Look at Ghost Normalization

Speech recognition with deep recurrent neural networks

Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton · 2013

Earlier work this paper cites.

Sequence to sequence learning with neural networks

Ilya Sutskever, Oriol Vinyals, and Quoc V Le · 2014

Earlier work this paper cites.

Batch normalization: Accelerating deep network training by reducing internal covariate shift

S. Ioffe and C. Szegedy · 2015

Earlier work this paper cites.

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015

Earlier work this paper cites.

Human-level control through deep reinforcement learning

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis · 2015

Earlier work this paper cites.

Densely connected convolutional networks

Original

G. Huang, Z. Liu, and Q. K. Weinberger · 2016

Earlier work this paper cites.

Mastering the game of go with deep neural networks and tree search

David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis · 2016

Earlier work this paper cites.

Layer Normalization

Original

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton · 2016

Earlier work this paper cites.

Instance Normalization: The Missing Ingredient for Fast Stylization

Original

Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky · 2016

Earlier work this paper cites.

Wide residual networks

Original

S. Zagoruyko and N. Komodakis · 2016

Cited alongside, same era.

ImageNet classification with deep convolutional neural networks

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton · 2017

Cited alongside, same era.

Batch renormalization: Towards reducing minibatch dependence in batch-normalized models

Sergey Ioffe · 2017

Cited alongside, same era.

Train longer, generalize better: Closing the generalization gap in large batch training of neural networks

Elad Hoffer, Itay Hubara, and Daniel Soudry · 2017

Cited alongside, same era.

Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates

Original

Leslie N. Smith and Nicholay Topin · 2017

Cited alongside, same era.

Group normalization

Y. Wu and K. He · 2018

Later among the works it cites.

Weight Standardization

Original

Siyuan Qiao, Huiyu Wang, Chenxi Liu, Wei Shen, and Alan Yuille · 2019

Later among the works it cites.

Differentiable learning-to-normalize via switchable normalization

Ping Luo, Jiamin Ren, Zhanglin Peng, Ruimao Zhang, and Jingyu Li · 2019

Later among the works it cites.

Exponential convergence rates for batch normalization: The power of length-direction decoupling in non-convex optimization

Jonas Kohler, Hadi Daneshmand, Aurelien Lucchi, Thomas Hofmann, Ming Zhou, and Klaus Neymeyr · 2019

Later among the works it cites.

RandAugment: Practical automated data augmentation with a reduced search space

Original

Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le · 2019

Later among the works it cites.