Fetching the paper…
Reading the bibliography…
In Continual Learning settings, deep neural networks are prone to Catastrophic Forgetting.
Catastrophic interference in connectionist networks: The sequential learning problem
Michael McCloskey and Neal J. Cohen · 1989
Earlier work this paper cites.
Connectionist models of recognition memory: constraints imposed by learning and forgetting functions
Roger Ratcliff · 1990
Earlier work this paper cites.
Catastrophic forgetting, rehearsal and pseudorehearsal
A. Robins · 1995
Earlier work this paper cites.
Avoiding catastrophic forgetting by coupling two reverberating neural networks
Bernard Ans and Stéphane Rousset · 1997
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann Lecun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
The connection between regularization operators and support vector kernels
Alex J. Smola, Bernhard Sch ̈o lkopf, and Klaus-Robert M ̈u ller · 1998
Earlier work this paper cites.
Catastrophic forgetting in connectionist networks
Robert French · 1999
Earlier work this paper cites.
Neural networks with a self-refreshing memory: Knowledge transfer in sequential learning tasks without catastrophic forgetting
Bernard Ans and Stéphane Rousset · 2000
Earlier work this paper cites.
Rademacher and gaussian complexities: Risk bounds and structural results
Peter L. Bartlett and Shahar Mendelson · 2003
Earlier work this paper cites.
Exploiting task relatedness for multiple task learning
Shai Ben-David and Reba Schuller · 2003
Earlier work this paper cites.
Curriculum learning
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
An empirical investigation of catastrophic forgeting in gradient-based neural networks
Ian J. Goodfellow, Mehdi Mirza, Xia Da, Aaron C. Courville, and Yoshua Bengio · 2014
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell · 2016
Earlier work this paper cites.
Regret Bounds for Lifelong Learning
Pierre Alquier, The Tien Mai, and Massimiliano Pontil · 2017
Earlier work this paper cites.
Kernel ridge vs. principal component regression: Minimax bounds and the qualification of regularization operators
Lee H. Dicker, Dean P. Foster, and Daniel Hsu · 2017
Earlier work this paper cites.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell · 2017
Earlier work this paper cites.
Gradient episodic memory for continuum learning
David Lopez-Paz and Marc’Aurelio Ranzato · 2017
Earlier work this paper cites.
Continual learning through synaptic intelligence
Friedemann Zenke, Ben Poole, and Surya Ganguli · 2017
Cited alongside, same era.
Memory aware synapses: Learning what (not) to forget
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars · 2018
Cited alongside, same era.
Don’t forget, there is more than forgetting: new metrics for continual learning
Natalia D ́i az Rodr ́i guez, Vincenzo Lomonaco, David Filliat, and Davide Maltoni · 2018
Cited alongside, same era.
Gradient descent provably optimizes over-parameterized neural networks
Simon S. Du, Xiyu Zhai, Barnab ́a s P ́o czos, and Aarti Singh · 2018
Cited alongside, same era.
Neural tangent kernel: Convergence and generalization in neural networks
Arthur Jacot, Franck Gabriel, and Cl ́e ment Hongler · 2018
Simple and effective regularization methods for training on noisily labeled data with generalization guarantee, 2019
Wei Hu, Zhiyuan Li, and Dingli Yu · 2019
Later among the works it cites.
Wide neural networks of any depth evolve as linear models under gradient descent, 2019
Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington · 2019
Later among the works it cites.
Mingchen Li, Mahdi Soltanolkotabi, and Samet Oymak · 2019
Later among the works it cites.
Towards understanding the transferability of deep representations, 2019
Hong Liu, Mingsheng Long, Jianmin Wang, and Michael I. Jordan · 2019
Later among the works it cites.
Toward understanding catastrophic forgetting in continual learning
Cuong V Nguyen, Alessandro Achille, Michael Lam, Tal Hassner, Vijay Mahadevan, and Stefano Soatto · 2019
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Lifelong learning with dynamically expandable networks
Jeongtae Lee, Jaehong Yoon, Eunho Yang, and Sung Ju Hwang · 2018
Cited alongside, same era.
Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
Nicolas Masse, Gregory Grant, and David Freedman · 2018
Cited alongside, same era.
Variational continual learning
Cuong V. Nguyen, Yingzhen Li, Thang D. Bui, and Richard E. Turner · 2018
Cited alongside, same era.
Online structured laplace approximations for overcoming catastrophic forgetting
Hippolyt Ritter, Aleksandar Botev, and David Barber · 2018
Cited alongside, same era.
Progress and compress: A scalable framework for continual learning, 05 2018
Jonathan Schwarz, Jelena Luketina, Wojciech Czarnecki, Agnieszka Grabska-Barwinska, Yee Teh, Razvan Pascanu, and Raia Hadsell · 2018
Cited alongside, same era.
Overcoming catastrophic forgetting with hard attention to the task
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou · 2018
Cited alongside, same era.
Generative replay with feedback connections as a general strategy for continual learning
Gido M Van de Ven and Andreas S Tolias · 2018
Cited alongside, same era.
Later among the works it cites.
Continual lifelong learning with neural networks: A review
German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter · 2019
Later among the works it cites.
Three scenarios for continual learning
Gido M Van de Ven and Andreas S Tolias · 2019
Later among the works it cites.
Regularization matters: Generalization and optimization of neural nets v.s. their induced kernel
Colin Wei, Jason D Lee, Qiang Liu, and Tengyu Ma · 2019
Later among the works it cites.
Understanding regularisation methods for continual learning
Frederik Benzing · 2020
Closest in time.
Few-shot learning via learning the representation, provably
Simon Shaolei Du, Wei Hu, Sham M. Kakade, Jason D. Lee, and Qi Lei · 2020
Closest in time.
Uncertainty-guided continual learning with bayesian neural networks
Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, and Marcus Rohrbach · 2020
Closest in time.
Modelling the influence of data structure on learning in neural networks, 2020
S. Goldt, M. M é zard, F. Krzakala, and L. Zdeborov á · 2020
Closest in time.
Multi-step model-agnostic meta-learning: Convergence and improved algorithms, 2020
Kaiyi Ji, Junjie Yang, and Yingbin Liang · 2020
Closest in time.
Continual learning with node-importance based adaptive group sparse regularization, 2020
Sangwon Jung, Hongjoon Ahn, Sungmin Cha, and Taesup Moon · 2020
Closest in time.
Optimal continual learning has perfect memory and is np-hard
Jeremias Knoblauch, H. Husain, and Tom Diethe · 2020
Closest in time.
Understanding the role of training regimes in continual learning, 2020
Seyed Iman Mirzadeh, Mehrdad Farajtabar, Razvan Pascanu, and Hassan Ghasemzadeh · 2020
Closest in time.
Continual deep learning by functional regularisation of memorable past, 2020
Pingbo Pan, Alexander Immer, Siddharth Swaroop, Runa Eschenhagen, Richard E Turner, and Mohammad Emtiyaz Khan · 2020
Closest in time.
A sample complexity separation between non-convex and convex meta-learning, 2020
Nikunj Saunshi, Yi Zhang, Mikhail Khodak, and Sanjeev Arora · 2020
Closest in time.
Is long horizon reinforcement learning more difficult than short horizon reinforcement learning?, 2020
Ruosong Wang, Simon S. Du, Lin F. Yang, and Sham M. Kakade · 2020
Closest in time.
Gradient surgery for multi-task learning, 2020
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn · 2020
Closest in time.