Fetching the paper…
Reading the bibliography…
Loss of plasticity is a phenomenon in which neural networks lose their ability to learn from new experience.
Cognitron: A self-organizing multilayered neural network
Fukushima, K · 1975
Earlier work this paper cites.
Optimal brain damage
LeCun, Y., Denker, J., and Solla, S · 1989
Earlier work this paper cites.
Continual learning in reinforcement environments
Ring, M. B · 1994
Earlier work this paper cites.
Flat minima
Hochreiter, S. and Schmidhuber, J · 1997
Earlier work this paper cites.
Lifelong learning algorithms
Thrun, S · 1998
Earlier work this paper cites.
Topmoumoute online natural gradient algorithm
Le Roux, N., Manzagol, P.-A., and Bengio, Y · 2007
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A · 2009
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Glorot, X. and Bengio, Y · 2010
Earlier work this paper cites.
MNIST handwritten digit database
LeCun, Y., Cortes, C., and Burges, C · 2010
Earlier work this paper cites.
Rectified linear units improve restricted Boltzmann machines
Nair, V. and Hinton, G. E · 2010
Earlier work this paper cites.
An empirical investigation of catastrophic forgetting in gradient-based neural networks
Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., and Bengio, Y · 2013
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Saxe, A., McClelland, J., and Ganguli, S · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2015
Earlier work this paper cites.
ImageNet large scale visual recognition challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L · 2015
Earlier work this paper cites.
Empirical evaluation of rectified activations in convolutional network
Xu, B., Wang, N., Chen, T., and Li, M · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
On large-batch training for deep learning: Generalization gap and sharp minima
Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P. T. P · 2016
Cited alongside, same era.
Understanding and improving convolutional neural networks via concatenated rectified linear units
Shang, W., Sohn, K., Almeida, D., and Lee, H · 2016
Cited alongside, same era.
Continual learning through synaptic intelligence
Zenke, F., Poole, B., and Ganguli, S · 2017
Cited alongside, same era.
Modern regularization methods for inverse problems
Benning, M. and Burger, M · 2018
Cited alongside, same era.
Gradient descent happens in a tiny subspace
Gur-Ari, G., Roberts, D. A., and Dyer, E · 2018
Cited alongside, same era.
Transient non-stationarity and generalisation in deep reinforcement learning
Igl, M., Farquhar, G., Luketina, J., Boehmer, W., and Whiteson, S · 2021
Later among the works it cites.
Implicit under-parameterization inhibits data-efficient deep reinforcement learning
Kumar, A., Agarwal, R., Ghosh, D., and Levine, S · 2021
Later among the works it cites.
Understanding and preventing capacity loss in reinforcement learning
Lyle, C., Rowland, M., and Dabney, W · 2021
Later among the works it cites.
On plasticity, invariance, and mutually frozen weights in sequential task learning
Zilly, J., Achille, A., Censi, A., and Frazzoli, E · 2021
Later among the works it cites.
Hesscale: Scalable computation of hessian diagonals
Elsayed, M. and Mahmood, A. R · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
One-dimensional empirical measures, order statistics, and Kantorovich transport distances , volume 261
Bobkov, S. and Ledoux, M · 2019
Cited alongside, same era.
Emergent properties of the local geometry of neural loss landscapes
Fort, S. and Ganguli, S · 2019
Cited alongside, same era.
Stiffness: A new perspective on generalization in neural networks
Fort, S., Nowak, P. K., Jastrzebski, S., and Narayanan, S · 2019
Cited alongside, same era.
An investigation into neural net optimization via hessian eigenvalue density
Ghorbani, B., Krishnan, S., and Xiao, Y · 2019
Cited alongside, same era.
Meta-learning representations for continual learning
Javed, K. and White, M · 2019
Cited alongside, same era.
Limitations of the empirical fisher approximation for natural gradient descent
Kunstner, F., Hennig, P., and Balles, L · 2019
Cited alongside, same era.
Harnessing structures for value-based planning and reinforcement learning
Yang, Y., Zhang, G., Xu, Z., and Katabi, D · 2019
Cited alongside, same era.
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de las Casas, D., Hendricks, L. A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Vinyals, O., Rae, J. W., and Sifre, L · 2022
Later among the works it cites.
Learning dynamics and generalization in deep reinforcement learning
Lyle, C., Rowland, M., Dabney, W., Kwiatkowska, M., and Gal, Y · 2022
Later among the works it cites.
The primacy bias in deep reinforcement learning
Nikishin, E., Schwarzer, M., D’Oro, P., Bacon, P.-L., and Courville, A · 2022
Later among the works it cites.
Loss of plasticity in continual deep reinforcement learning
Abbas, Z., Zhao, R., Modayil, J., White, A., and Machado, M. C · 2023
Closest in time.
Utility-based perturbed gradient descent: An optimizer for continual learning
Elsayed, M. and Mahmood, A. R · 2023
Closest in time.
Maintaining plasticity via regenerative regularization
Kumar, S., Marklund, H., and Roy, B. V · 2023
Closest in time.
Understanding plasticity in neural networks
Lyle, C., Zheng, Z., Nikishin, E., Avila Pires, B., Pascanu, R., and Dabney, W · 2023
Closest in time.
The dormant neuron phenomenon in deep reinforcement learning
Sokar, G., Agarwal, R., Castro, P. S., and Evci, U · 2023
Closest in time.
Symmetry leads to structured constraint of learning
Ziyin, L · 2023
Closest in time.
Learning Continually by Spectral Regularization
Lewandowski, A., Kumar, S., Schuurmans, D., György, A., and Machado, M. C · 2024
Closest in time.