Fetching the paper…
Reading the bibliography…
Plasticity, the ability of a neural network to quickly change its predictions in response to new information, is essential for the adaptability and robustness of deep reinforcement learning systems.
Stiffness: A new perspective on generalization in neural networks
Fort, S., Nowak, P. K., and Narayanan, S · 1901
Earlier work this paper cites.
On the uniform convergence of frequencies of occurrence of events to their probabilities
Vapnik, V · 1968
Earlier work this paper cites.
Catastrophic forgetting in connectionist networks
French, R. M · 1999
Earlier work this paper cites.
Synaptic plasticity: taming the beast
Abbott, L. F. and Nelson, S. B · 2000
Earlier work this paper cites.
Rademacher and gaussian complexities: Risk bounds and structural results
Bartlett, P. L. and Mendelson, S · 2002
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Glorot, X. and Bengio, Y · 2010
Earlier work this paper cites.
The mnist database of handwritten digit images for machine learning research
Deng, L · 2012
Earlier work this paper cites.
Deep learning without shortcuts: Shaping the kernel with tailored rectifiers
Zhang, G., Botev, A., and Martens, J · 2012
Earlier work this paper cites.
The arcade learning environment: An evaluation platform for general agents
Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M · 2013
Earlier work this paper cites.
The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects, 2013
Mermillod, M., Bugaiska, A., and Bonin, P · 2013
Earlier work this paper cites.
On the importance of initialization and momentum in deep learning
Sutskever, I., Martens, J., Dahl, G., and Hinton, G · 2013
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., and Sun, J · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2015
Earlier work this paper cites.
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al · 2015
Earlier work this paper cites.
Ba, J. L., Kiros, J. R., and Hinton, G. E · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
Exponential expressivity in deep neural networks through transient chaos
Poole, B., Lahiri, S., Raghu, M., Sohl-Dickstein, J., and Ganguli, S · 2016
Earlier work this paper cites.
Deep reinforcement learning with double q-learning
Van Hasselt, H., Guez, A., and Silver, D · 2016
Earlier work this paper cites.
Dueling network architectures for deep reinforcement learning
Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N · 2016
Earlier work this paper cites.
The shattered gradients problem: If resnets are the answer, then what is the question?
Balduzzi, D., Frean, M., Leary, L., Lewis, J., Ma, K. W.-D., and McWilliams, B · 2017
Earlier work this paper cites.
Sharp minima can generalize for deep nets
Dinh, L., Pascanu, R., Bengio, S., and Bengio, Y · 2017
Cited alongside, same era.
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al · 2017
Cited alongside, same era.
Overcoming catastrophic forgetting by incremental moment matching
Lee, S.-W., Kim, J.-H., Jun, J., Ha, J.-W., and Zhang, B.-T · 2017
Cited alongside, same era.
Deep information propagation
Schoenholz, S. S., Gilmer, J., Ganguli, S., and Sohl-Dickstein, J · 2017
Cited alongside, same era.
Mean field residual networks: On the edge of chaos
Yang, G. and Schoenholz, S · 2017
Cited alongside, same era.
Measuring catastrophic forgetting in neural networks
Kemker, R., McClure, M., Abitino, A., Hayes, T., and Kanan, C · 2018
Cited alongside, same era.
The break-even point on optimization trajectories of deep neural networks
Jastrzebski, S., Szymczak, M., Fort, S., Arpit, D., Tabor, J., Cho*, K., and Geras*, K · 2020
Later among the works it cites.
Scaling laws for neural language models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D · 2020
Later among the works it cites.
Implicit under-parameterization inhibits data-efficient deep reinforcement learning
Kumar, A., Agarwal, R., Ghosh, D., and Levine, S · 2020
Later among the works it cites.
The large learning rate phase of deep learning: the catapult mechanism
Lewkowycz, A., Bahri, Y., Dyer, E., Sohl-Dickstein, J., and Gur-Ari, G · 2020
Later among the works it cites.
DQN Zoo: Reference implementations of DQN-based agents, 2020
Quan, J. and Ostrovski, G · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Visualizing the loss landscape of neural nets
Li, H., Xu, Z., Taylor, G., Studer, C., and Goldstein, T · 2018
Cited alongside, same era.
How does batch normalization help optimization?
Santurkar, S., Tsipras, D., Ilyas, A., and Madry, A · 2018
Cited alongside, same era.
Kickstarting deep reinforcement learning
Schmitt, S., Hudson, J. J., Zidek, A., Osindero, S., Doersch, C., Czarnecki, W. M., Leibo, J. Z., Kuttler, H., Zisserman, A., Simonyan, K., et al · 2018
Cited alongside, same era.
Reinforcement learning: An introduction
Sutton, R. S. and Barto, A. G · 2018
Cited alongside, same era.
An investigation into neural net optimization via hessian eigenvalue density
Ghorbani, B., Krishnan, S., and Xiao, Y · 2019
Cited alongside, same era.
Experience replay for continual learning
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., and Wayne, G · 2019
Cited alongside, same era.
Toward training recurrent neural networks for lifelong learning
Sodhani, S., Chandar, S., and Bengio, Y · 2020
Later among the works it cites.
A study on the plasticity of neural networks
Berariu, T., Czarnecki, W., De, S., Bornschein, J., Smith, S., Pascanu, R., and Clopath, C · 2021
Later among the works it cites.
Gradient descent on neural networks typically occurs at the edge of stability
Cohen, J. M., Kaur, S., Li, Y., Kolter, J. Z., and Talwalkar, A · 2021
Later among the works it cites.
Adaptive rational activations to boost deep reinforcement learning
Delfosse, Q., Schramowski, P., Mundt, M., Molina, A., and Kersting, K · 2021
Later among the works it cites.
Continual backprop: Stochastic gradient descent with persistent randomness
Dohare, S., Mahmood, A. R., and Sutton, R. S · 2021
Later among the works it cites.
Spectral normalisation for deep reinforcement learning: an optimisation perspective
Gogianu, F., Berariu, T., Rosca, M. C., Clopath, C., Busoniu, L., and Pascanu, R · 2021
Later among the works it cites.
Transient non-stationarity and generalisation in deep reinforcement learning
Igl, M., Farquhar, G., Luketina, J., Boehmer, W., and Whiteson, S · 2021
Later among the works it cites.
Understanding and preventing capacity loss in reinforcement learning
Lyle, C., Rowland, M., and Dabney, W · 2021
Later among the works it cites.
Martens, J., Ballard, A., Desjardins, G., Swirszcz, G., Dalibard, V., Sohl-Dickstein, J., and Schoenholz, S. S · 2021
Later among the works it cites.
A loss curvature perspective on training instability in deep learning
Gilmer, J., Ghorbani, B., Garg, A., Kudugunta, S. R., Neyshabur, B., Cardoze, D., Dahl, G. E., Nado, Z., and Firat, O · 2022
Later among the works it cites.
An empirical study of implicit regularization in deep offline rl
Gulcehre, C., Srinivasan, S., Sygnowski, J., Ostrovski, G., Farajtabar, M., Hoffman, M., Pascanu, R., and Doucet, A · 2022
Later among the works it cites.
Learning dynamics and generalization in deep reinforcement learning
Lyle, C., Rowland, M., Dabney, W., Kwiatkowska, M., and Gal, Y · 2022
Later among the works it cites.
The primacy bias in deep reinforcement learning
Nikishin, E., Schwarzer, M., D’Oro, P., Bacon, P.-L., and Courville, A · 2022
Later among the works it cites.
How do vision transformers work?
Park, N. and Kim, S · 2022
Later among the works it cites.
A universal law of robustness via isoperimetry
Bubeck, S. and Sellke, M · 2023
Closest in time.