FReLU: Flexible Rectified Linear Units for Improving Convolutional Neural Networks

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation , vol. 9, no. 8, pp. 1735–1780, 1997

1997

Earlier work this paper cites.

C. J. B. Yann LeCun, Corinna Cortes, “The mnist database of handwritten digits,” http://yann.lecun.com/exdb/mnist/ , 1998

1998

Earlier work this paper cites.

A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” 2009

2009

Earlier work this paper cites.

V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th international conference on machine learning (ICML-10) , 2010, pp. 807–814

2010

Earlier work this paper cites.

X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , 2010, pp. 249–256

2010

Earlier work this paper cites.

X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks.” in Aistats , vol. 15, no. 106, 2011, p. 275

2011

Earlier work this paper cites.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems , 2012, pp. 1097–1105

2012

Earlier work this paper cites.

A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” in Proc. ICML , vol. 30, no. 1, 2013

2013

Earlier work this paper cites.

M. Lin, Q. Chen, and S. Yan, “Network in network,” arXiv preprint arXiv:1312.4400 , 2013

Original

2013

Earlier work this paper cites.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 1026–1034

2015

Cited alongside, same era.

B. Xu, N. Wang, T. Chen, and M. Li, “Empirical evaluation of rectified activations in convolutional network,” arXiv preprint arXiv:1505.00853 , 2015

Original

2015

Cited alongside, same era.

D.-A. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and accurate deep network learning by exponential linear units (elus),” arXiv preprint arXiv:1511.07289 , 2015

Original

2015

Cited alongside, same era.

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167 , 2015

Original

2015

Cited alongside, same era.

Y. Li, C. Fan, Y. Li, and Q. Wu, “Improving deep neural network with multiple parametric exponential linear units,” arXiv preprint arXiv:1606.00305 , 2016

Original

2016

Later among the works it cites.

B. Xu, R. Huang, and M. Li, “Revise saturated activation functions,” arXiv preprint arXiv:1602.05980 , 2016

Original

2016

Later among the works it cites.

S. Gross and M. Wilber, “Training and investigating residual nets,” Facebook AI Research, CA.[Online]. Avilable: http://torch. ch/blog/2016/02/04/resnets. html , 2016

2016

Later among the works it cites.

A. Shah, E. Kadam, H. Shah, and S. Shinde, “Deep residual networks with exponential linear unit,” arXiv preprint arXiv:1604.04112 , 2016

Original

2016

Later among the works it cites.

B. Carlile, G. Delamarter, P. Kinney, A. Marti, and B. Whitney, “Improving deep learning by inverse square root linear units (isrlus),” arXiv preprint arXiv:1710.09967 , 2017