Fetching the paper…
Reading the bibliography…
The scope of research in the domain of activation functions remains limited and centered around improving the ease of optimization or generalization quality of neural networks (NNs).
Learning representations by back-propagating errors
David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams · 1986
Earlier work this paper cites.
Universal approximation bounds for superpositions of a sigmoidal function
A. R. Barron · 1993
Earlier work this paper cites.
Efficient backprop
Yann LeCun, Léon Bottou, Genevieve B. Orr, and Klaus-Robert Müller · 1998
Earlier work this paper cites.
Scikit-learn: Machine learning in Python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay · 2011
Earlier work this paper cites.
Even mirror fourier nonlinear filters
A. Carini and G. L. Sicuranza · 2013
Earlier work this paper cites.
Maxout networks
Ian Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio · 2013
Earlier work this paper cites.
Learning polynomials with neural networks
Alexandr Andoni, Rina Panigrahy, Gregory Valiant, and Li Zhang · 2014
Earlier work this paper cites.
Saga: A fast incremental gradient method with support for non-strongly convex composite objectives
Aaron Defazio, Francis Bach, and Simon Lacoste-Julien · 2014
Cited alongside, same era.
Min Lin, Qiang Chen, and Shuicheng Yan · 2014
Cited alongside, same era.
Learning activation functions to improve deep neural networks
Forest Agostinelli, Matthew D. Hoffman, Peter J. Sadowski, and Pierre Baldi · 2015
Cited alongside, same era.
The Loss Surfaces of Multilayer Networks
Anna Choromanska, MIkael Henaff, Michael Mathieu, Gerard Ben Arous, and Yann LeCun · 2015
Cited alongside, same era.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Neural network with unbounded activations is universal approximator
Sho Sonoda and Noboru Murata · 2015
Later among the works it cites.
Fast and accurate deep network learning by exponential linear units (elus)
Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter · 2016
Later among the works it cites.
Taming the waves: sine as activation function in deep neural networks
T. Virtanen G. Parascandolo, H. Huttunen · 2017
Later among the works it cites.
ConvNets with Smooth Adaptive Activation Functions for Regression
Le Hou, Dimitris Samaras, Tahsin Kurc, Yi Gao, and Joel Saltz · 2017
Later among the works it cites.
Searching for activation functions
Prajit Ramachandran, Barret Zoph, and Quoc V. Le · 2017
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Cited alongside, same era.
Weight sharing is crucial to succesful optimization
Shai Shalev-Shwartz, Ohad Shamir, and Shaked Shammah · 2017
Later among the works it cites.