TanhExp: A Smooth Activation Function with High Convergence Speed for Lightweight Neural Networks

Srivastava, N., Hinton, G., Krizhevsky, A.: ’Dropout: A simple way to prevent neural networks from overfitting’, Journal of Machine Learning Research

1958

Earlier work this paper cites.

Lecun, Y., Bottou, L., Bengio, Y., et al. : ’Gradient-based learning applied to document recognition’, Proceedings of the IEEE

1998

Earlier work this paper cites.

Krizhevsky, A.: ’Learning multiple layers of features from tiny images’. Master’s thesis, University of Toronto, 2009

2009

Earlier work this paper cites.

Nair, V., Hinton, G.: ’ Rectified linear units improve restricted boltzmann machines’, International Conference on Machine Learning

2010

Earlier work this paper cites.

Glorot, X., Bordes, A., Bengio, Y.: ’Deep sparse rectifier neural networks’, Proceedings of the fourteenth international conference on artificial intelligence and statistics

2011

Earlier work this paper cites.

Krizhevsky, A., Sutskever, I., Hinton, G.: ’Imagenet classification with deep convolutional neural networks’, Advances in Neural Information Processing Systems

2012

Earlier work this paper cites.

Hannun, A., Maas, A., Ng, A.:’Rectifier nonlinearities improve neural network acoustic models’, ICML Workshop on Deep Learning for Audio, Speech and Language Processing

2013

Earlier work this paper cites.

He, K., Zhang, X., Ren, S., et al. : ’Delving deep into rectifiers: Surpassing human-level performance on imagenet classification’, IEEE International Conference on Computer Vision (ICCV)

2015

Earlier work this paper cites.

’Empirical evaluation of rectified activations in convolutional network’, https://arxiv.org/abs/1505.00853, accessed 5 May 2015

Original

2015

Earlier work this paper cites.

’Batch normalization: Accelerating deep network training by reducing internal covariate shift’, http://www.arxiv.org/abs/1502.03167, accessed 11 February 2015

Original

2015

Earlier work this paper cites.

Redmon, J., Divvala, S., Girshick, R., et al. : "You Only Look Once: Unified, Real-Time Object Detection", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

2016

Earlier work this paper cites.

Held, D., Thrun, S., Savarese, S.: ’Learning to Track at 100 FPS with Deep Regression Networks’, European Conference on Computer Vision

2016

Cited alongside, same era.

Clevert, D., Unterthiner, T., Hochreiter, S.: ’Fast and accurate deep network learning by exponential linear units (elus)’, International Conference on Learning Representations

2016

Cited alongside, same era.

Jin, X., Xu, C., Feng, X., et al. : ’Deep learning with s-shaped rectified linear activation unit’, Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence

2016

Cited alongside, same era.

’Bridging nonlinearities and stochastic regularizers with gaussian error linear units’, http://arxiv.org/abs/1606.08415v1, accessed 27 June 2016

Original

2016

Cited alongside, same era.

He, K., Zhang, X., Ren, S., et al. : ’Deep Residual Learning for Image Recognition’, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Sandler, M., Howard, A., Zhu, M., et al. : "MobileNetV2: Inverted Residuals and Linear Bottlenecks", 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

2018

Later among the works it cites.

’BERT: pre-training of deep bidirectional transformers for language understanding’, https://arxiv.org/abs/1810.04805, accessed 11 October 2018

Original

2018

Later among the works it cites.

’Deep Learning for Classical Japanese Literature’, https://arxiv.org/abs/1812.01718, accessed 3 December 2018

Original

2018

Later among the works it cites.

Zhang, X., Zhou, X., Lin, M., et al. : ’Shufflenet: An extremely efficient convolutional neural network for mobile devices’, 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

2018

Later among the works it cites.