Parametric Exponential Linear Unit for Deep Convolutional Neural Networks

V. Nair and G. Hinton, “Rectified linear units improve restricted boltzmann machines,” in ICML , 2010, pp. 807–814

Original

2010

Earlier work this paper cites.

A. Krizhevsky, I. Sutskever, and G. Hinton, “Imagenet classification with deep convolutional neural networks,” in NIPS , 2012, pp. 1097–1105

2012

Earlier work this paper cites.

N. Srivastava and R. R. Salakhutdinov, “Multimodal learning with deep boltzmann machines,” in NIPS , 2012, pp. 2222–2230

2012

Earlier work this paper cites.

G. H. Tijmen Tieleman, “Lecture 6.5 - rmsprop: Divide the gradient by a running average of its recent magnitude.” COURSERA: Neural Networks for Machine Learning , vol. 4, 2012

2012

Earlier work this paper cites.

M. Lin, Q. Chen, and S. Yan, “Network in network,” arXiv preprint arXiv:1312.4400 , 2013

Original

2013

Earlier work this paper cites.

P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, “Overfeat: Integrated recognition, localization and detection using convolutional networks,” arXiv preprint arXiv:1312.6229 , 2013

Original

2013

Earlier work this paper cites.

A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” in ICML Workshop on Deep Learning for Audio, Speech and Language Processing . Citeseer, 2013

2013

Earlier work this paper cites.

I. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and Y. Bengio, “Maxout networks,” in ICML , 2013, pp. 1319–1327

2013

Earlier work this paper cites.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

Original

2014

Cited alongside, same era.

F. Agostinelli, M. Hoffman, P. Sadowski, and P. Baldi, “Learning activation functions to improve deep neural networks,” arXiv preprint arXiv:1412.6830 , 2014

Original

2014

Cited alongside, same era.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” JMLR , vol. 15, no. 1, pp. 1929–1958, 2014

2014

Cited alongside, same era.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature , vol. 521, no. 7553, pp. 436–444, 2015

2015

Cited alongside, same era.

O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, “Show and tell: A neural image caption generator,” in CVPR , 2015, pp. 3156–3164

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in ICML , 2015, pp. 448–456

2015

Later among the works it cites.

J. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller, “Striving for simplicity: The all convolutional net,” in ICLR (workshop track) , 2015

2015

Later among the works it cites.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in ICCV , 2015, pp. 1026–1034

2015

Later among the works it cites.

X. Jin, C. Xu, J. Feng, Y. Wei, J. Xiong, and S. Yan, “Deep learning with s-shaped rectified linear activation units,” arXiv preprint arXiv:1512.07030 , 2015

Original

2015

Later among the works it cites.

J. Hosang, R. Benenson, P. Dollár, and B. Schiele, “What makes for effective detection proposals?” PAMI , vol. 38, no. 4, pp. 814–830, 2016