Fetching the paper…
Reading the bibliography…
Continual learning of partially similar tasks poses a challenge for artificial neural networks, as task similarity presents both an opportunity for knowledge transfer and a risk of interference and catastrophic forgetting.
Three unfinished works on the optimal storage capacity of networks
E. Gardner and B. Derrida · 1989
Earlier work this paper cites.
Catastrophic interference in connectionist networks: The sequential learning problem
M. McCloskey and N. J. Cohen · 1989
Earlier work this paper cites.
Using semi-distributed representations to overcome catastrophic forgetting in connectionist networks
R. M. French · 1991
Earlier work this paper cites.
Statistical mechanics of learning from examples
H. S. Seung, H. Sompolinsky, and N. Tishby · 1992
Earlier work this paper cites.
Catastrophic forgetting, rehearsal and pseudorehearsal
A. Robins · 1995
Earlier work this paper cites.
On-line learning in soft committee machines
D. Saad and S. A. Solla · 1995
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
Modelling time perception in rats: evidence for catastrophic interference in animal learning
R. French and A. Ferrara · 1999
Earlier work this paper cites.
Catastrophic forgetting in connectionist networks
R. M. French · 1999
Earlier work this paper cites.
Learning curves for stochastic gradient descent in linear feedforward networks
J. Werfel, X. Xie, and H. Seung · 2003
Earlier work this paper cites.
An empirical investigation of catastrophic forgetting in gradient-based neural networks
I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, and Y. Bengio · 2013
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
A. M. Saxe, J. L. McClelland, and S. Ganguli · 2013
Earlier work this paper cites.
Compete to compute
R. K. Srivastava, J. Masci, S. Kazerounian, F. Gomez, and J. Schmidhuber · 2013
Earlier work this paper cites.
The benefit of multitask representation learning
A. Maurer, M. Pontil, and B. Romera-Paredes · 2016
Earlier work this paper cites.
A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell · 2016
Earlier work this paper cites.
Statistical physics of inference: Thresholds and algorithms
L. Zdeborová and F. Krzakala · 2016
Earlier work this paper cites.
A theory of multineuronal dimensionality, dynamics and measurement
P. Gao, E. Trautmann, B. Yu, G. Santhanam, S. Ryu, K. Shenoy, and S. Ganguli · 2017
Earlier work this paper cites.
Overcoming catastrophic forgetting in neural networks
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al · 2017
Earlier work this paper cites.
Overcoming catastrophic forgetting by incremental moment matching
S.-W. Lee, J.-H. Kim, J. Jun, J.-W. Ha, and B.-T. Zhang · 2017
Earlier work this paper cites.
An analytical formula of population gradient for two-layered relu network and its applications in convergence and critical point analysis
Y. Tian · 2017
Earlier work this paper cites.
Lifelong learning with dynamically expandable networks
J. Yoon, E. Yang, J. Lee, and S. J. Hwang · 2017
Earlier work this paper cites.
On compressing deep models by low rank and sparse decomposition
X. Yu, T. Liu, X. Wang, and D. Tao · 2017
Earlier work this paper cites.
Continual learning through synaptic intelligence
F. Zenke, B. Poole, and S. Ganguli · 2017
Cited alongside, same era.
Comparing dynamics: Deep neural networks versus glassy systems
M. Baity-Jesi, L. Sagun, M. Geiger, S. Spigler, G. B. Arous, C. Cammarota, Y. LeCun, M. Wyart, and G. Biroli · 2018
Cited alongside, same era.
An analytic theory of generalization dynamics and transfer learning in deep linear networks
A. K. Lampinen and S. Ganguli · 2018
Cited alongside, same era.
Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
N. Y. Masse, G. D. Grant, and D. J. Freedman · 2018
Cited alongside, same era.
Overcoming catastrophic forgetting with hard attention to the task
J. Serra, D. Suris, M. Miron, and A. Karatzoglou · 2018
Cited alongside, same era.
Brain-inspired replay for continual learning with artificial neural networks
G. M. Van de Ven, H. T. Siegelmann, and A. S. Tolias · 2020
Later among the works it cites.
Statistical mechanical analysis of catastrophic forgetting in continual learning with teacher and student networks
H. Asanuma, S. Takagi, Y. Nagano, Y. Yoshida, Y. Igarashi, and M. Okada · 2021
Later among the works it cites.
Phase transitions in transfer learning for high-dimensional perceptrons
O. Dhifallah and Y. M. Lu · 2021
Later among the works it cites.
A theoretical analysis of catastrophic forgetting through the ntk overlap matrix
T. Doan, M. A. Bennani, B. Mazoure, G. Rabusseau, and P. Alquier · 2021
Later among the works it cites.
Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity
M. Jazayeri and S. Ostojic · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
A. R. Zamir, A. Sax, W. Shen, L. J. Guibas, J. Malik, and S. Savarese · 2018
Cited alongside, same era.
Limitations of lazy training of two-layers neural network
B. Ghorbani, S. Mei, T. Misiakiewicz, and A. Montanari · 2019
Cited alongside, same era.
Continual learning via neural pruning
S. Golkar, M. Kagan, and K. Cho · 2019
Cited alongside, same era.
Generalization in multitask deep neural classifiers: a statistical physics approach
A. Ndirango and T. Lee · 2019
Cited alongside, same era.
Experience replay for continual learning
D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne · 2019
Cited alongside, same era.
A mathematical theory of semantic development in deep neural networks
A. M. Saxe, J. L. McClelland, and S. Ganguli · 2019
Cited alongside, same era.
High-dimensional dynamics of generalization error in neural networks
M. S. Advani, A. M. Saxe, and H. Sompolinsky · 2020
Cited alongside, same era.
Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting
R. Karakida and S. Akaho · 2021
Later among the works it cites.
Continual learning in the teacher-student setup: Impact of task similarity
S. Lee, S. Goldt, and A. Saxe · 2021
Later among the works it cites.
A rapid and efficient learning rule for biological neural circuits
E. Sezener, A. Grabska-Barwińska, D. Kostadinov, M. Beau, S. Krishnagopal, D. Budden, M. Hutter, J. Veness, M. Botvinick, C. Clopath, et al · 2021
Later among the works it cites.
Memory bounds for continual learning
X. Chen, C. Papadimitriou, and B. Peng · 2022
Later among the works it cites.
How catastrophic can catastrophic forgetting be in linear regression?
I. Evron, E. Moroshko, R. Ward, N. Srebro, and D. Soudry · 2022
Later among the works it cites.
Provable continual learning via sketched jacobian approximations
R. Heckel · 2022
Later among the works it cites.
On the stability and scalability of node perturbation learning
N. Hiratani, Y. Mehta, T. Lillicrap, and P. E. Latham · 2022
Later among the works it cites.
Biological underpinnings for lifelong learning machines
D. Kudithipudi, M. Aguilar-Simon, J. Babb, M. Bazhenov, D. Blackiston, J. Bongard, A. P. Brna, S. Chakravarthi Raja, N. Cheney, J. Clune, et al · 2022
Later among the works it cites.
Beyond not-forgetting: Continual learning with backward knowledge transfer
S. Lin, L. Yang, D. Fan, and J. Zhang · 2022
Later among the works it cites.
On the statistical benefits of curriculum learning
Z. Xu and A. Tewari · 2022
Later among the works it cites.
Analysis of catastrophic forgetting for random orthogonal transformation tasks in the overparameterized regime
D. Goldfarb and P. Hand · 2023
Later among the works it cites.
Sub-network discovery and soft-masking for continual learning of mixed tasks
Z. Ke, B. Liu, W. Xiong, A. Celikyilmaz, and H. Li · 2023
Later among the works it cites.
Statistical mechanics of continual learning: Variational principle and mean-field potential
C. Li, Z. Huang, W. Zou, and H. Huang · 2023
Later among the works it cites.
An empirical study of catastrophic forgetting in large language models during continual fine-tuning
Y. Luo, Z. Yang, F. Meng, Y. Li, J. Zhou, and Y. Zhang · 2023
Later among the works it cites.
Incorporating neuro-inspired adaptability for continual learning in artificial intelligence
L. Wang, X. Zhang, Q. Li, M. Zhang, H. Su, J. Zhu, and Y. Zhong · 2023
Later among the works it cites.
I. Evron, D. Goldfarb, N. Weinberger, D. Soudry, and P. Hand · 2024
Closest in time.