Fetching the paper…
Reading the bibliography…
Recent studies have shown that the choice of activation function can significantly affect the performance of deep learning networks.
Delta-Coding: An iterative search strategy for genetic algorithms
D. Whitley, K. Mathias, and P. Fitzhorn · 1991
Earlier work this paper cites.
Genetic programming: on the programming of computers by means of natural selection , volume 1
J. R. Koza · 1992
Earlier work this paper cites.
Evolving neural networks through augmenting topologies
K. O. Stanley and R. Miikkulainen · 2002
Earlier work this paper cites.
Active guidance for a finless rocket using neuroevolution
F. Gomez and R. Miikkulainen · 2003
Earlier work this paper cites.
Distributed computing in practice: the condor experience
D. Thain, T. Tannenbaum, and M. Livny · 2005
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and F.-F. Li · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky, G. Hinton, et al · 2009
Earlier work this paper cites.
Rectified linear units improve restricted boltzmann machines
V. Nair and G. E. Hinton · 2010
Earlier work this paper cites.
Improving neural networks by preventing co-adaptation of feature detectors
G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov · 2012
Earlier work this paper cites.
Rectifier nonlinearities improve neural network acoustic models
A. L. Maas, A. Y. Hannun, and A. Y. Ng · 2013
Earlier work this paper cites.
Learning activation functions to improve deep neural networks
F. Agostinelli, M. Hoffman, P. Sadowski, and P. Baldi · 2014
Earlier work this paper cites.
Fast and accurate deep network learning by exponential linear units (elus)
D.-A. Clevert, T. Unterthiner, and S. Hochreiter · 2015
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
K. He, X. Zhang, S. Ren, and J. Sun · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
S. Ioffe and C. Szegedy · 2015
Cited alongside, same era.
Striving for simplicity: The all convolutional net
J. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller · 2015
Cited alongside, same era.
Tensorflow: A system for large-scale machine learning
M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al · 2016
Cited alongside, same era.
Gaussian error linear units (gelus)
D. Hendrycks and K. Gimpel · 2016
Cited alongside, same era.
S. Zagoruyko and N. Komodakis · 2016
Cited alongside, same era.
Self-normalizing neural networks
Y. Li, C. Wei, and T. Ma · 2019
Later among the works it cites.
Mish: A self regularized non-monotonic neural activation function
D. Misra · 2019
Later among the works it cites.
A. Molina, P. Schramowski, and K. Kersting · 2019
Later among the works it cites.
Regularized evolution for image classifier architecture search
E. Real, A. Aggarwal, Y. Huang, and Q. V. Le · 2019
Later among the works it cites.
Efficientnet: Rethinking model scaling for convolutional neural networks
M. Tan and Q. Le · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
G. Klambauer, T. Unterthiner, A. Mayr, and S. Hochreiter · 2017
Cited alongside, same era.
The quest for the golden activation function
M. Basirat and P. M. Roth · 2018
Cited alongside, same era.
Sigmoid-weighted linear units for neural network function approximation in reinforcement learning
S. Elfwing, E. Uchibe, and K. Doya · 2018
Cited alongside, same era.
Activation functions: Comparison of trends in practice and research for deep learning
C. Nwankpa, W. Ijomah, A. Gachagan, and S. Marshall · 2018
Cited alongside, same era.
Searching for activation functions
P. Ramachandran, B. Zoph, and Q. V. Le · 2018
Cited alongside, same era.
Neural architecture search: A survey
T. Elsken, J. H. Metzen, and F. Hutter · 2019
Cited alongside, same era.
Improved training speed, accuracy, and data utilization through loss function optimization
S. Gonzalez and R. Miikkulainen · 2019
Cited alongside, same era.
A survey on neural architecture search
M. Wistuba, A. Rawat, and T. Pedapati · 2019
Later among the works it cites.
Evolutionary optimization of deep learning activation functions
G. Bingham, W. Macke, and R. Miikkulainen · 2020
Closest in time.
Evolving loss functions with multivariate taylor polynomial parameterizations
S. Gonzalez and R. Miikkulainen · 2020
Closest in time.
Population-based training for loss function optimization
J. Liang, S. Gonzalez, and R. Miikkulainen · 2020
Closest in time.
Evolving normalization-activation layers
H. Liu, A. Brock, K. Simonyan, and Q. V. Le · 2020
Closest in time.
Splash: Learnable activation functions for improving accuracy and adversarial robustness
M. Tavakoli, F. Agostinelli, and P. Baldi · 2020
Closest in time.
C. Xie, M. Tan, B. Gong, A. Yuille, and Q. V. Le · 2020
Closest in time.