Fetching the paper…
Reading the bibliography…
We propose two approaches of locally adaptive activation functions namely, layer-wise and neuron-wise locally adaptive activation functions, which improve the performance of deep and physics-informed neural networks.
H. Bateman, Some recent researches on the motion of fluids, Monthly Weather Review, 43(4), 163-170, 1915
1915
Earlier work this paper cites.
J.M. Burgers,A mathematical model illustrating the theory of turbulence. In Advances in Applied Mechanics, Vol. 1, pp. 171-199, 1948
1948
Earlier work this paper cites.
C. Basdevant, et al., Spectral and finite difference solution of the Burgers equation, Comput. Fluids, 14 (1986) 23-41
1986
Earlier work this paper cites.
Tactile Srl, Brescia, Italy (1994). Semeion Handwritten Digit Data Set. Semeion Research Center of Sciences of Communication, Rome, Italy
1994
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278-2324, 1998
1998
Earlier work this paper cites.
D.P. Bertsekas, Nonlinear programming, Athena scientific Belmont, 1999
1999
Earlier work this paper cites.
C. Yu, et al.,An adaptive activation function for multilayer feedforward neural networks, 2002 IEEE Region 10 Conference on Computers, Communications, Control and Power Engineering. TENCOM ’02. Proceedings
2002
Earlier work this paper cites.
Y. Shen, B. Wang, F. Chen and L. Cheng, A new multi-output neural model with tunable activation function and its applications, Neural Processing Letters, 20: 85-104, 2004
2004
Earlier work this paper cites.
A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009
2009
Earlier work this paper cites.
G.B. Whitham, Linear and nonlinear waves, Vol. 42, John-Wiley & Sons, 2011
2011
Earlier work this paper cites.
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, Andrew Y. Ng Reading digits in natural images with unsupervised feature learning, NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011
2011
Cited alongside, same era.
G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, B. Kingsbury, et al. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal processing magazine, 29, 2012
2012
Cited alongside, same era.
A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097-1105, 2012
2012
Cited alongside, same era.
B. Li, Y. Li and X. Rong, The extreme learning machine learning algorithm with tunable activation function, Neural Comput & Applie (2013) 22: 531-539
2013
Cited alongside, same era.
A.G. Baydin, B.A. Pearlmutter, A.A. Radul, J.M. Siskind, Automatic differentiation in machine learning: a survey, Journal of Machine Learning Research, 18 (2018) 1-43
2018
Later among the works it cites.
A.D. Jagtap, Method of relaxed streamline upwinding for hyperbolic conservation laws, Wave Motion, Vol. 78 (2018) 132-161
2018
Later among the works it cites.
S. Qian, et al, Adaptive activation functions in convolutional neural networks, Neurocomputing Volume 272, 10 January 2018, Pages 204-212
2018
Later among the works it cites.
V. Kunc and J. Kl e ´ \acute{\text{e}} ma, On transformative adaptive activation functions in neural networks for gene expression inference, bioRxiv, doi:http://dx.doi.org/10.1101/587287, 2019
2019
Closest in time.
M. Raissi, P. Perdikaris, G.E. Karniadakis, Physics-informed neural network: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys., 378, 686-707, 2019
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, Journal of Machine Learning Research, 15(Jun):1929-1958, 2014
2014
Cited alongside, same era.
K. He, X. Zhang, S. Ren, J. Sun, Identity mappings in deep residual networks, European conference on computer vision, pp. 630-645, 2016
2016
Cited alongside, same era.
2016
Cited alongside, same era.
D. P. Kingma, J. L. Ba, ADAM: A method for stochastic optimization, arXiv:1412.6980v9, 2017
2017
Cited alongside, same era.
S. Ruder, An overview of gradient descent optimization algorithms, arXiv:1609.04747v2, 2017
2017
Cited alongside, same era.
2017
Cited alongside, same era.
M. Dushkoff, R. Ptucha, Adaptive Activation Functions for Deep Networks, Electronic Imaging, Computational Imaging XIV, pp. 1-5(5)
Cited in the paper.
Cited in the paper.
2019
Closest in time.
A.D. Jagtap, K. Kawaguchi and G.E. Karniadakis, Adaptive activation functions accelerate convergence in deep and physics-informed neural networks, J. Comput. Phys., 404 (2020) 109136
2020
Closest in time.
A.D. Jagtap, E. Kharazmi and G.E. Karniadakis, Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems, Computer Methods in Applied Mechanics and Engineering, 365 (2020) 113028
2020
Closest in time.
2020
Closest in time.