Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting

M. McCloskey and N. J. Cohen, “Catastrophic interference in connectionist networks: The sequential learning problem,” Psychology of learning and motivation , vol. 24, pp. 109–165, 1989

1989

Earlier work this paper cites.

S.-I. Amari, “Natural gradient works efficiently in learning,” Neural computation , vol. 10, no. 2, pp. 251–276, 1998

1998

Earlier work this paper cites.

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998

1998

Earlier work this paper cites.

D. Silver and R. Mercer, “The task rehearsal method of life-long learning: Overcoming impoverished data,” Advances in Artificial Intelligence , pp. 90–101, 2002

2002

Earlier work this paper cites.

T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM review , vol. 51, no. 3, pp. 455–500, 2009

2009

Earlier work this paper cites.

A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” Tech. Rep., 2009

2009

Earlier work this paper cites.

C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The Caltech-UCSD Birds-200-2011 Dataset,” no. CNS-TR-2011-001, 2011

2011

Earlier work this paper cites.

B. Yao, X. Jiang, A. Khosla, A. L. Lin, L. Guibas, and L. Fei-Fei, “Human action recognition by learning bases of action attributes and parts,” in ICCV , 2011, pp. 1331–1338

2011

Earlier work this paper cites.

R. Pascanu and Y. Bengio, “Revisiting natural gradient for deep networks,” arXiv preprint arXiv:1301.3584 , 2013

Original

2013

Earlier work this paper cites.

R. K. Srivastava, J. Masci, S. Kazerounian, F. Gomez, and J. Schmidhuber, “Compete to compute,” in NIPS , 2013, pp. 2310–2318

2013

Earlier work this paper cites.

J. Li, “Restructuring of deep neural network acoustic models with singular value decomposition,” in Interspeech , January 2013

2013

Cited alongside, same era.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” CoRR , vol. abs/1409.1556, 2014

Original

2014

Cited alongside, same era.

Z. Li and D. Hoiem, “Learning without forgetting,” in ECCV , 2016, pp. 614–629

2016

Cited alongside, same era.

R. Grosse and J. Martens, “A kronecker-factored approximate fisher matrix for convolution layers,” in ICML , 2016, pp. 573–582

2016

Cited alongside, same era.

S.-A. Rebuffi, A. Kolesnikov, and C. H. Lampert, “iCaRL: Incremental classifier and representation learning,” in CVPR , 2017

2017

Cited alongside, same era.

R. Aljundi, P. Chakravarty, and T. Tuytelaars, “Expert gate: Lifelong learning with a network of experts,” in CVPR , 2017

2017

Later among the works it cites.

H. Shin, J. K. Lee, J. Kim, and J. Kim, “Continual learning with deep generative replay,” in Advances in Neural Information Processing Systems , 2017, pp. 2994–3003

2017

Later among the works it cites.

A. Rannen, R. Aljundi, and M. B. B. T. Tuytelaars, “Encoder based lifelong learning,” in CVPR , 2017, pp. 1320–1328

2017

Later among the works it cites.

J. Lafond, N. Vasilache, and L. Bottou, “Diagonal rescaling for neural networks,” arXiv preprint arXiv:1705.09319 , 2017

Original

2017

Later among the works it cites.

M. Masana, J. van de Weijer, L. Herranz, A. D. Bagdanov, and J. M. Alvarez, “Domain-adaptive deep network compression,” in International conference on Computer Vision (ICCV) , 2017