Fetching the paper…
Reading the bibliography…
Neural networks have shown tremendous growth in recent years to solve numerous problems.
M. Kobayashi, Singularities of three-layered complex-valued neural networks with split activation function, IEEE Transactions on Neural Networks and Learning Systems 29 (5) (2017) 1900–1907
1907
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324
1998
Earlier work this paper cites.
W. Duch, N. Jankowski, Survey of neural transfer functions, Neural Computing Surveys 2 (1) (1999) 163–212
1999
Earlier work this paper cites.
C. Dugas, Y. Bengio, F. Bélisle, C. Nadeau, R. Garcia, Incorporating second-order functional knowledge for better option pricing, in: Advances in Neural Information Processing Systems, 2001, pp. 472–478
2001
Earlier work this paper cites.
K. Papineni, S. Roukos, T. Ward, W.-J. Zhu, Bleu: a method for automatic evaluation of machine translation, in: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318
2002
Earlier work this paper cites.
P. Chandra, Y. Singh, An activation function adapting training algorithm for sigmoidal feedforward networks, Neurocomputing 61 (2004) 429–437
2004
Earlier work this paper cites.
Y. Liu, J. Zhang, C. Gao, J. Qu, L. Ji, Natural-logarithm-rectified activation function in convolutional neural networks, in: International Conference on Computer and Communications, 2019, pp. 2000–2008
2008
Earlier work this paper cites.
A. Krizhevsky, Learning multiple layers of features from tiny images, Tech Report, Univ. of Toronto (2009)
2009
Earlier work this paper cites.
V. Nair, G. E. Hinton, Rectified linear units improve restricted boltzmann machines, in: International Conference on Machine Learning, 2010, pp. 807–814
2010
Earlier work this paper cites.
X. Glorot, A. Bordes, Y. Bengio, Deep sparse rectifier neural networks, in: International Conference on Artificial Intelligence and Statistics, 2011, pp. 315–323
2011
Earlier work this paper cites.
B. Karlik, A. V. Olgac, Performance analysis of various activation functions in generalized mlp architectures of neural networks, International Journal of Artificial Intelligence and Expert Systems 1 (4) (2011) 111–122
2011
Earlier work this paper cites.
C. H. Dagli, Artificial neural networks for intelligent manufacturing, Springer Science & Business Media, 2012
2012
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, 2012, pp. 1097–1105
2012
Earlier work this paper cites.
A. Graves, A.-r. Mohamed, G. Hinton, Speech recognition with deep recurrent neural networks, in: IEEE International Conference on Acoustics, Speech and Signal Processing, 2013, pp. 6645–6649
2013
Earlier work this paper cites.
A. L. Maas, A. Y. Hannun, A. Y. Ng, Rectifier nonlinearities improve neural network acoustic models, in: International Conference on Machine Learning, Vol. 30, 2013, p. 3
2013
Earlier work this paper cites.
2013
Earlier work this paper cites.
S. S. Sodhi, P. Chandra, Bi-modal derivative activation function for sigmoidal feedforward networks, Neurocomputing 143 (2014) 182–196
2014
Earlier work this paper cites.
2014
Earlier work this paper cites.
F. Agostinelli, M. Hoffman, P. Sadowski, P. Baldi, Learning activation functions to improve deep neural networks, International Conference on Learning Representations Workshops (2015)
2015
Earlier work this paper cites.
Y. LeCun, Y. Bengio, G. Hinton, Deep learning, nature 521 (7553) (2015) 436–444
2015
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, in: IEEE international conference on computer vision, 2015, pp. 1026–1034
2015
Earlier work this paper cites.
2015
Earlier work this paper cites.
L. B. Godfrey, M. S. Gashler, A continuum among logarithmic, linear, and exponential functions, and its potential to improve generalization in neural networks, in: International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, Vol. 1, 2015, pp. 481–486
2015
Earlier work this paper cites.
H. Zheng, Z. Yang, W. Liu, J. Liang, Y. Li, Improving deep neural networks using softplus units, in: International Joint Conference on Neural Networks, 2015, pp. 1–4
2015
Earlier work this paper cites.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in: IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9
2015
Earlier work this paper cites.
A. N. S. Njikam, H. Zhao, A novel activation function for multilayer feed-forward neural networks, Applied Intelligence 45 (1) (2016) 75–82
2016
Earlier work this paper cites.
B. Xu, R. Huang, M. Li, Revise saturated activation functions, International Conference on Learning Representations Workshop (2016)
2016
Earlier work this paper cites.
D.-A. Clevert, T. Unterthiner, S. Hochreiter, Fast and accurate deep network learning by exponential linear units (elus), in: International Conference on Learning Representations, 2016
2016
Earlier work this paper cites.
W. Shang, K. Sohn, D. Almeida, H. Lee, Understanding and improving convolutional neural networks via concatenated rectified linear units, in: International Conference on Machine Learning, 2016, pp. 2217–2225
2016
Earlier work this paper cites.
S. S. Liew, M. Khalil-Hani, R. Bakhteri, Bounded activation functions for enhanced training stability of deep neural networks on visual pattern recognition problems, Neurocomputing 216 (2016) 718–734
2016
Earlier work this paper cites.
C. Gulcehre, M. Moczulski, M. Denil, Y. Bengio, Noisy activation functions, in: International Conference on Machine Learning, 2016, pp. 3059–3068
2016
Earlier work this paper cites.
H. Li, W. Ouyang, X. Wang, Multi-bias non-linear activation in deep neural networks, in: International Conference on Machine Learning, 2016, pp. 221–229
2016
Earlier work this paper cites.
X. Jin, C. Xu, J. Feng, Y. Wei, J. Xiong, S. Yan, Deep learning with s-shaped rectified linear activation units, in: AAAI Conference on Artificial Intelligence, 2016
2016
Earlier work this paper cites.
M. Dushkoff, R. Ptucha, Adaptive activation functions for deep networks, Electronic Imaging 2016 (19) (2016) 1–5
2016
Earlier work this paper cites.
Q. Liu, S. Furber, Noisy softplus: a biology inspired activation function, in: International Conference on Neural Information Processing, 2016, pp. 405–412
2016
Earlier work this paper cites.
D. Hendrycks, K. Gimpel, Gaussian error linear units (gelus), arXiv preprint arXiv:1606.08415 (2016)
2016
Earlier work this paper cites.
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778
2016
Earlier work this paper cites.
S. Kong, M. Takatsuka, Hexpo: A vanishing-proof activation function, in: International Joint Conference on Neural Networks, 2017, pp. 2562–2567
2017
Earlier work this paper cites.
R. Duggal, A. Gupta, P-telu: Parametric tan hyperbolic linear unit activation for deep neural networks, in: IEEE International Conference on Computer Vision Workshops, 2017, pp. 974–978
2017
Earlier work this paper cites.
G. Klambauer, T. Unterthiner, A. Mayr, S. Hochreiter, Self-normalizing neural networks, in: Advances in Neural Information Processing Systems, 2017, pp. 971–980
2017
Earlier work this paper cites.
J. T. Barron, Continuously differentiable exponential linear units, arXiv (2017) arXiv–1704
2017
Earlier work this paper cites.
L. Trottier, P. Gigu, B. Chaib-draa, et al., Parametric exponential linear unit for deep convolutional neural networks, in: IEEE International Conference on Machine Learning and Applications, 2017, pp. 207–214
2017
Earlier work this paper cites.
S. Scardapane, M. Scarpiniti, D. Comminiello, A. Uncini, Learning activation functions from data using cubic spline interpolation, in: Italian Workshop on Neural Nets, 2017, pp. 73–83
2017
Earlier work this paper cites.
A. Mishra, P. Chandra, U. Ghose, S. S. Sodhi, Bi-modal derivative adaptive activation function sigmoidal feedforward artificial neural networks, Applied Soft Computing 61 (2017) 983–994
2017
Earlier work this paper cites.
C. Eisenach, Z. Wang, H. Liu, Nonparametrically learning activation functions in deep neural nets, in: International Conference on Learning Representations Workshops, 2017
2017
Earlier work this paper cites.
C. J. Vercellino, W. Y. Wang, Hyperactivations for activation function exploration, in: Conference on Neural Information Processing Systems Workshop on Meta-learning, 2017
2017
Earlier work this paper cites.
Q. Su, L. Carin, et al., A probabilistic framework for nonlinearities in stochastic neural networks, in: Advances in Neural Information Processing Systems, 2017, pp. 4486–4495
2017
Earlier work this paper cites.
L. Hou, D. Samaras, T. M. Kurc, Y. Gao, J. H. Saltz, Convnets with smooth adaptive activation functions for regression, Proceedings of Machine Learning Research 54 (2017) 430
2017
Earlier work this paper cites.
M. Telgarsky, Neural networks and rational functions, in: International Conference on Machine Learning, 2017, pp. 3387–3393
2017
Earlier work this paper cites.
J. Pennington, S. Schoenholz, S. Ganguli, Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice, in: Advances in Neural Information Processing Systems, 2017, pp. 4785–4795
2017
Earlier work this paper cites.
2017
Earlier work this paper cites.
D. Yarotsky, Error bounds for approximations with deep relu networks, Neural Networks 94 (2017) 103–114
2017
Cited alongside, same era.
2017
Cited alongside, same era.
H. K. Vydana, A. K. Vuppala, Investigative study of various activation functions for speech recognition, in: National Conference on Communications, 2017, pp. 1–5
2017
Cited alongside, same era.
2017
Cited alongside, same era.
2019
Later among the works it cites.
D. Klabjan, M. Harmon, Activation ensembles for deep neural networks, in: IEEE International Conference on Big Data, 2019, pp. 206–214
2019
Later among the works it cites.
K. Sun, J. Yu, L. Zhang, Z. Dong, A convolutional neural network model based on improved softplus activation function, in: International Conference on Applications and Techniques in Cyber Security and Intelligence, 2019, pp. 1326–1335
2019
Later among the works it cites.
Y. Chen, Y. Mai, J. Xiao, L. Zhang, Improving the antinoise ability of dnns via a bio-inspired noise adaptive activation function rand softplus, Neural Computation 31 (6) (2019) 1215–1233
2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2017
Cited alongside, same era.
Y. Qin, X. Wang, J. Zou, The optimized deep belief networks with improved logistic sigmoid units and their application in fault diagnosis for planetary gearboxes of wind turbines, IEEE Transactions on Industrial Electronics 66 (5) (2018) 3814–3824
2018
Cited alongside, same era.
S. Elfwing, E. Uchibe, K. Doya, Sigmoid-weighted linear units for neural network function approximation in reinforcement learning, Neural Networks 107 (2018) 3–11
2018
Cited alongside, same era.
P. Ramachandran, B. Zoph, Q. V. Le, Searching for activation functions, International Conference on Learning Representations Workshops (2018)
2018
Cited alongside, same era.
S. Qiu, X. Xu, B. Cai, Frelu: Flexible rectified linear units for improving convolutional neural networks, in: International Conference on Pattern Recognition, 2018, pp. 1223–1228
2018
Cited alongside, same era.
X. Jiang, Y. Pang, X. Li, J. Pan, Y. Xie, Deep neural networks with elastic rectified linear units for object recognition, Neurocomputing 275 (2018) 1132–1139
2018
Cited alongside, same era.
J. Cao, Y. Pang, X. Li, J. Liang, Randomly translational activation inspired by the input distributions of relu, Neurocomputing 275 (2018) 859–868
2018
Cited alongside, same era.
F. Godin, J. Degrave, J. Dambre, W. De Neve, Dual rectified linear units (drelus): A replacement for tanh activation functions in quasi-recurrent neural networks, Pattern Recognition Letters 116 (2018) 8–14
2018
Cited alongside, same era.
2019
Later among the works it cites.
2019
Later among the works it cites.
2019
Later among the works it cites.
E. López-Rubio, F. Ortega-Zamorano, E. Domínguez, J. Muñoz-Pérez, Piecewise polynomial activation functions for feedforward neural networks, Neural Processing Letters 50 (1) (2019) 121–147
2019
Later among the works it cites.
2019
Later among the works it cites.
2019
Later among the works it cites.
A. Apicella, F. Isgrò, R. Prevete, A simple and efficient architecture for trainable activation functions, Neurocomputing 370 (2019) 1–15
2019
Later among the works it cites.
A. Asif, et al., Learning neural activations, arXiv preprint arXiv:1912.12187 (2019)
2019
Later among the works it cites.
S. Scardapane, S. Van Vaerenbergh, S. Totaro, A. Uncini, Kafnets: Kernel-based non-parametric activation functions for neural networks, Neural Networks 110 (2019) 19–32
2019
Later among the works it cites.
S. Scardapane, E. Nieddu, D. Firmani, P. Merialdo, Multikernel activation functions: formulation and a case study, in: INNS Big Data and Deep Learning conference, 2019, pp. 320–329
2019
Later among the works it cites.
S. Scardapane, S. Van Vaerenbergh, D. Comminiello, A. Uncini, Widely linear kernels for complex-valued kernel activation functions, in: IEEE International Conference on Acoustics, Speech and Signal Processing, 2019, pp. 8528–8532
2019
Later among the works it cites.
2019
Later among the works it cites.
2019
Later among the works it cites.
D. Aguirre, O. Fuentes, Improving weight initialization of relu and output layers, in: International Conference on Artificial Neural Networks, 2019, pp. 170–184
2019
Later among the works it cites.
R. Burkholz, A. Dubatovka, Initialization of relus for dynamical isometry, in: Advances in Neural Information Processing Systems, 2019, pp. 2382–2392
2019
Later among the works it cites.
M. Hein, M. Andriushchenko, J. Bitterwolf, Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem, in: IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 41–50
2019
Later among the works it cites.
S. Goel, S. Karmalkar, A. Klivans, Time/accuracy tradeoffs for learning a relu with respect to gaussian marginals, in: Advances in Neural Information Processing Systems, 2019, pp. 8582–8591
2019
Later among the works it cites.
S. Dittmer, J. Emily, P. Maass, Singular values for relu layers, IEEE Transactions on Neural Networks and Learning Systems (2019)
2019
Later among the works it cites.
K. Eckle, J. Schmidt-Hieber, A comparison of deep networks with relu activation function and linear spline-type methods, Neural Networks 110 (2019) 232–242
2019
Later among the works it cites.
A. K. Dubey, V. Jain, Comparative study of convolution neural network’s relu and leaky-relu activation functions, in: Applications of Computing, Automation and Wireless Systems in Electrical Engineering, Springer, 2019, pp. 873–880
2019
Later among the works it cites.
C. Banerjee, T. Mukherjee, E. Pasiliao Jr, An empirical study on generalizations of the relu activation function, in: ACM Southeast Conference, 2019, pp. 164–167
2019
Later among the works it cites.
2019
Later among the works it cites.
G. Castaneda, P. Morris, T. M. Khoshgoftaar, Evaluation of maxout activations in deep learning across several big data domains, Journal of Big Data 6 (1) (2019) 72
2019
Later among the works it cites.
K. K. Babu, S. R. Dubey, Pcsgan: Perceptual cyclic-synthesized generative adversarial networks for thermal and nir to visible image transformation, Neurocomputing (2020)
2020
Later among the works it cites.
S. S. Basha, S. R. Dubey, V. Pulabaigari, S. Mukherjee, Impact of fully connected layers on performance of convolutional neural networks for image classification, Neurocomputing 378 (2020) 112–119
2020
Later among the works it cites.
2020
Later among the works it cites.
M. Basirat, P. Roth, L* relu: Piece-wise linear activation functions for deep fine-grained visual categorization, in: IEEE Winter Conference on Applications of Computer Vision, 2020, pp. 1218–1227
2020
Later among the works it cites.
D. Kim, J. Kim, J. Kim, Elastic exponential linear units for convolutional neural networks, Neurocomputing 406 (2020) 253–266
2020
Later among the works it cites.
Q. Cheng, H. Li, Q. Wu, L. Ma, N. N. King, Parametric deformable exponential linear units for deep neural networks, Neural Networks 125 (2020) 281–289
2020
Later among the works it cites.
Y. Yu, K. Adu, N. Tashi, P. Anokye, X. Wang, M. A. Ayidzoe, Rmaf: Relu-memristor-like activation function for deep learning, IEEE Access 8 (2020) 72727–72741
2020
Later among the works it cites.
A. D. Jagtap, K. Kawaguchi, G. E. Karniadakis, Adaptive activation functions accelerate convergence in deep and physics-informed neural networks, Journal of Computational Physics 404 (2020) 109136
2020
Later among the works it cites.
2020
Later among the works it cites.
A. Molina, P. Schramowski, K. Kersting, Pad e ´ \acute{e} activation units: End-to-end learning of flexible activation functions in deep networks, International Conference on Learning Representations (2020)
2020
Later among the works it cites.
2020
Later among the works it cites.
2020
Later among the works it cites.
M. Wang, B. Liu, H. Foroosh, Wide hidden expansion layer for deep convolutional neural networks, in: IEEE Winter Conference on Applications of Computer Vision, 2020, pp. 934–942
2020
Later among the works it cites.
2020
Later among the works it cites.
Y. Wang, Y. Li, Y. Song, X. Rong, The influence of the activation function in a convolution neural network model of facial expression recognition, Applied Sciences 10 (5) (2020) 1897
2020
Later among the works it cites.
2020
Later among the works it cites.
T. Szandała, Review and comparison of commonly used activation functions for deep neural networks, in: Bio-inspired Neurocomputing, 2020, pp. 203–224
2020
Later among the works it cites.
S. R. Dubey, A decade survey of content based image retrieval using deep learning, IEEE Transactions on Circuits and Systems for Video Technology (2021)
2021
Closest in time.
H. Li, Y. Pan, J. Zhao, L. Zhang, Skin disease diagnosis with deep learning: a review, Neurocomputing 464 (2021) 364–393
2021
Closest in time.
F. Shao, L. Chen, J. Shao, W. Ji, S. Xiao, L. Ye, Y. Zhuang, J. Xiao, Deep learning for weakly-supervised object detection and localization: A survey, Neurocomputing (2022)
2022
Closest in time.
Y. Mo, Y. Wu, X. Yang, F. Liu, Y. Liao, Review the state-of-the-art technologies of semantic segmentation based on deep learning, Neurocomputing (2022)
2022
Closest in time.
Y. Guo, F. Feng, X. Hao, X. Chen, Jac-net: Joint learning with adaptive exploration and concise attention for unsupervised domain adaptive person re-identification, Neurocomputing (2022)
2022
Closest in time.
X. Xia, X. Pan, N. Li, X. He, L. Ma, X. Zhang, N. Ding, Gan-based anomaly detection: A review, Neurocomputing (2022)
2022
Closest in time.
J. Liu, Y. Liu, Q. Zhang, A weight initialization method based on neural network with asymmetric activation function, Neurocomputing (2022)
2022
Closest in time.