Fetching the paper…
Reading the bibliography…
Loss of plasticity is a phenomenon where neural networks can become more difficult to train over the course of learning.
Continual learning in reinforcement environments
Mark Bishop Ring · 1994
Earlier work this paper cites.
Tracking the best expert
M. Herbster and M. K. Warmuth · 1998
Earlier work this paper cites.
MNIST handwritten digit database
Yann LeCun, Corinna Cortes, and CJ Burges · 1998
Earlier work this paper cites.
Lifelong learning algorithms
Sebastian Thrun · 1998
Earlier work this paper cites.
Eigenvalue computation in the 20th century
Gene H Golub and Henk A Van der Vorst · 2000
Earlier work this paper cites.
Online convex programming and generalized infinitesimal gradient ascent
M. Zinkevich · 2003
Earlier work this paper cites.
Reducing the dimensionality of data with neural networks
Geoffrey E Hinton and Ruslan R Salakhutdinov · 2006
Earlier work this paper cites.
The effective rank: A measure of effective dimensionality
Olivier Roy and Martin Vetterli · 2007
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio · 2010
Earlier work this paper cites.
Double Q-learning
Hado Hasselt · 2010
Earlier work this paper cites.
Reading digits in natural images with unsupervised feature learning
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al · 2011
Earlier work this paper cites.
Matrix theory: basic results and techniques
Fuzhen Zhang · 2011
Earlier work this paper cites.
Matrix computations
Gene H Golub and Charles F Van Loan · 2013
Earlier work this paper cites.
The theory of matrices in numerical analysis
Alston S Householder · 2013
Earlier work this paper cites.
Intriguing properties of neural networks
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus · 2013
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
A Saxe, J McClelland, and S Ganguli · 2014
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Adam: A Method for Stochastic Optimization
Diederik P. Kingma and Jimmy Ba · 2015
Earlier work this paper cites.
Tiny imagenet visual recognition challenge
Ya Le and Xuan Yang · 2015
Earlier work this paper cites.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton · 2016
Earlier work this paper cites.
Shifting regret, mirror descent, and matrices
András György and Csaba Szepesvári · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Understanding and improving convolutional neural networks via concatenated rectified linear units
Wenling Shang, Kihyuk Sohn, Diogo Almeida, and Honglak Lee · 2016
Cited alongside, same era.
Emnist: Extending mnist to handwritten letters
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik · 2017
Cited alongside, same era.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al · 2017
Cited alongside, same era.
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
Jeffrey Pennington, Samuel Schoenholz, and Surya Ganguli · 2017
Cited alongside, same era.
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms
Han Xiao, Kashif Rasul, and Roland Vollgraf · 2017
Cited alongside, same era.
Direction matters: On the implicit bias of stochastic gradient descent with moderate learning rate
Jingfeng Wu, Difan Zou, Vladimir Braverman, and Quanquan Gu · 2021
Later among the works it cites.
On plasticity, invariance, and mutually frozen weights in sequential task learning
Julian Zilly, Alessandro Achille, Andrea Censi, and Emilio Frazzoli · 2021
Later among the works it cites.
Sample-Efficient Reinforcement Learning by Breaking the Replay Ratio Barrier
Pierluca D’Oro, Max Schwarzer, Evgenii Nikishin, Pierre-Luc Bacon, Marc G. Bellemare, and Aaron Courville · 2022
Later among the works it cites.
Rank diminishing in deep neural networks
Ruili Feng, Kecheng Zheng, Yukun Huang, Deli Zhao, Michael Jordan, and Zheng-Jun Zha · 2022
Later among the works it cites.
Characterizing the Implicit Bias of Regularized SGD in Rank Minimization
Tomer Galanti, Zachary S. Siegel, Aparna Gupte, and Tomaso Poggio · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Yuichi Yoshida and Takeru Miyato · 2017
Cited alongside, same era.
Continual learning through synaptic intelligence
Friedemann Zenke, Ben Poole, and Surya Ganguli · 2017
Cited alongside, same era.
Modern regularization methods for inverse problems
Martin Benning and Martin Burger · 2018
Cited alongside, same era.
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, 2018
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine · 2018
Cited alongside, same era.
Spectral normalization for generative adversarial networks
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida · 2018
Cited alongside, same era.
The emergence of spectral universality in deep networks
Jeffrey Pennington, Samuel Schoenholz, and Surya Ganguli · 2018
Cited alongside, same era.
Lipschitz-margin training: Scalable certification of perturbation invariance for deep neural networks
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama · 2018
Cited alongside, same era.
Real-time rideshare driver supply values using online reinforcement learning
Benjamin Han, Hyungjun Lee, and Sébastien Martin · 2022
Later among the works it cites.
An empirical analysis of compute-optimal large language model training
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Oriol Vinyals, Jack William Rae, and Laurent Sifre · 2022
Later among the works it cites.
Understanding and preventing capacity loss in reinforcement learning
Clare Lyle, Mark Rowland, and Will Dabney · 2022
Later among the works it cites.
The primacy bias in deep reinforcement learning
Evgenii Nikishin, Max Schwarzer, Pierluca D’Oro, Pierre-Luc Bacon, and Aaron Courville · 2022
Later among the works it cites.
Three types of incremental learning
Gido M Van de Ven, Tinne Tuytelaars, and Andreas S. Tolias · 2022
Later among the works it cites.
Loss of plasticity in continual deep reinforcement learning
Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, and Marlos C. Machado · 2023
Later among the works it cites.
Efficient Online Reinforcement Learning with Offline Data, 2023
Philip J. Ball, Laura Smith, Ilya Kostrikov, and Sergey Levine · 2023
Later among the works it cites.
Implicit bias of large depth networks: a notion of rank for nonlinear functions
Arthur Jacot · 2023
Later among the works it cites.
Gvfs in the real world: making predictions online for water treatment
Muhammad Kamran Janjua, Haseeb Shah, Martha White, Erfan Miahi, Marlos C Machado, and Adam White · 2023
Later among the works it cites.
Maintaining plasticity via regenerative regularization
Saurabh Kumar, Henrik Marklund, and Benjamin Van Roy · 2023
Later among the works it cites.
Directions of Curvature as an Explanation for Loss of Plasticity
Alex Lewandowski, Haruto Tanaka, Dale Schuurmans, and Marlos C. Machado · 2023
Later among the works it cites.
Understanding plasticity in neural networks
Clare Lyle, Zeyu Zheng, Evgenii Nikishin, Bernardo Avila Pires, Razvan Pascanu, and Will Dabney · 2023
Later among the works it cites.
The dormant neuron phenomenon in deep reinforcement learning
Ghada Sokar, Rishabh Agarwal, Pablo Samuel Castro, and Utku Evci · 2023
Later among the works it cites.
Implicit regularization towards rank minimization in relu networks
Nadav Timor, Gal Vardi, and Ohad Shamir · 2023
Later among the works it cites.
Loss of plasticity in deep continual learning
Shibhansh Dohare, J. Fernando Hernandez-Garcia, Qingfeng Lan, Parash Rahman, A. Rupam Mahmood, and Richard S. Sutton · 2024
Closest in time.
Addressing loss of plasticity and catastrophic forgetting in continual learning
Mohamed Elsayed and A. Rupam Mahmood · 2024
Closest in time.
Disentangling the Causes of Plasticity Loss in Neural Networks
Clare Lyle, Zeyu Zheng, Khimya Khetarpal, Hado van Hasselt, Razvan Pascanu, James Martens, and Will Dabney · 2024
Closest in time.
Tuning for the Unknown: Revisiting Evaluation Strategies for Lifelong RL
Golnaz Mesbahi, Olya Mastikhina, Parham Mohammad Panahi, Martha White, and Adam White · 2024
Closest in time.
Overestimation, Overfitting, and Plasticity in Actor-Critic: The Bitter Lesson of Reinforcement Learning, 2024
Michal Nauman, Michał Bortkiewicz, Piotr Miłoś, Tomasz Trzciński, Mateusz Ostaszewski, and Marek Cygan · 2024
Closest in time.