Residual Connections Encourage Iterative Inference

Speed of processing in the human visual system

S. Thorpe, D. Fize, and C. Marlot · 1996

Earlier work this paper cites.

Long short-term memory

S. Hochreiter and J. Schmidhuber · 1997

Earlier work this paper cites.

Learning multiple layers of features from tiny images

A. Krizhevsky and G. Hinton · 2009

Earlier work this paper cites.

Understanding the difficulty of training deep feedforward neural networks

X. Glorot and Y. Bengio · 2010

Earlier work this paper cites.

Rectified linear units improve restricted boltzmann machines

V. Nair and G. Hinton · 2010

Earlier work this paper cites.

Imagenet classification with deep convolutional neural networks

A. Krizhevsky, I. Sutskever, and G. Hinton · 2012

Earlier work this paper cites.

Very deep convolutional networks for large-scale image recognition

K. Simonyan and A. Zisserman · 2014

Cited alongside, same era.

Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

K. He, X. Zhang, S. Ren, and J. Sun · 2015

Cited alongside, same era.

Batch normalization: Accelerating deep network training by reducing internal covariate shift

S. Ioffe and C. Szegedy · 2015

Cited alongside, same era.

Normalization propagation: A parametric technique for removing internal covariate shift in deep networks

D. Arpit, Y. Zhou, B. U Kota, and V. Govindaraju · 2016

Cited alongside, same era.

Recurrent batch normalization

Original

Tim Cooijmans, Nicolas Ballas, César Laurent, Çağlar Gülçehre, and Aaron Courville · 2016

Cited alongside, same era.

Recurrent orthogonal networks and long-memory tasks

M. Henaff, A. Szlam, and Y. LeCun · 2016

Later among the works it cites.

Bridging the gaps between residual learning, recurrent neural networks and visual cortex

Q. Liao and T. Poggio · 2016

Later among the works it cites.

The loss surface of residual networks: Ensembles and the role of batch normalization

E. Littwin and L. Wolf · 2016

Later among the works it cites.

The time-course of ultrarapid categorization: The influence of scene congruency and top-down processing

S. Vanmarcke, F. Calders, and F. Wagemans · 2016

Later among the works it cites.

Residual networks are exponential ensembles of relatively shallow networks

A. Veit, M. Wilber, and S. Belongie · 2016

Later among the works it cites.

Learning deep resnet blocks sequentially using boosting theory