Built on

Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., Fu, Y.: Large scale incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 374–382 (2019), https://arxiv.org/abs/1905.13260
Original
1905
Earlier work this paper cites.
Traoré, R., Caselles-Dupré, H., Lesort, T., Sun, T., Cai, G., Rodríguez, N.D., Filliat, D.: Discorl: Continual reinforcement learning via policy distillation. CoRR abs/1907.05855
Original
1907
Earlier work this paper cites.
Lange, M.D., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: Continual learning: A comparative study on how to defy forgetting in classification tasks (2019), https://arxiv.org/abs/1909.08383
Original
1909
Earlier work this paper cites.
Farajtabar, M., Azizan, N., Mott, A., Li, A.: Orthogonal gradient descent for continual learning (2019), https://arxiv.org/abs/1910.07104
Original
1910
Earlier work this paper cites.
Caccia, L., Belilovsky, E., Caccia, M., Pineau, J.: Online learned continual compression with adaptative quantization module. arXiv preprint arXiv:1911.08019 (2019), https://arxiv.org/abs/1911.08019
Original
1911
Earlier work this paper cites.
French, R.M.: Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences 3
1999
Earlier work this paper cites.
van der Maaten, L., Hinton, G.: Visualizing data using t-sne (2008)
2008
Earlier work this paper cites.
LeCun, Y., Cortes, C.: MNIST handwritten digit database. public (2010), http://yann.lecun.com/exdb/mnist/
2010
Earlier work this paper cites.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python (2011)
2011
Earlier work this paper cites.
Rusu, A.A., Rabinowitz, N.C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., Hadsell, R.: Progressive neural networks. ArXiv e-prints (jun 2016), https://arxiv.org/abs/1606.04671
Original
2016
Earlier work this paper cites.
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proc. of the national academy of sciences (2017), https://www.pnas.org/content/pnas/114/13/3521.full.pdf
2017
Earlier work this paper cites.

Similar

Then

Hou, S., Pan, X., Loy, C.C., Wang, Z., Lin, D.: Learning a unified classifier incrementally via rebalancing. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)
2019
Closest in time.
Lesort, T., Caselles-Dupré, H., Garcia-Ortiz, M., Goudou, J.F., Filliat, D.: Generative models from the perspective of continual learning. In: IJCNN - International Joint Conference on Neural Networks. Budapest, Hungary (Jul 2019), https://hal.archives-ouvertes.fr/hal-01951954
2019
Closest in time.
Lesort, T., Gepperth, A., Stoian, A., Filliat, D.: Marginal replay vs conditional replay for continual learning. In: International Conference on Artificial Neural Networks. pp. 466–480. Springer (2019), https://arxiv.org/abs/1810.12069
Original
2019
Closest in time.
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., Díaz-Rodríguez, N.: Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges. Information Fusion 58
2019

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2019

Regularization Shortcomings for Continual Learning

Lesort, Timothée, Stoian, Andrei, Filliat, David

Understand

In most machine learning algorithms, training data is assumed to be independent and identically distributed (iid).

When it is not the case, the algorithm's performances are challenged, leading to the famous phenomenon of catastrophic forgetting.
Algorithms dealing with it are gathered in the Continual Learning research field.
In this paper, we study the regularization based approaches to continual learning and show that those approaches can not learn to discriminate classes from different tasks in an elemental continual benchmark: the class-incremental scenario.

Built on

Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., Fu, Y.: Large scale incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 374–382 (2019), https://arxiv.org/abs/1905.13260
Original
1905
Earlier work this paper cites.
Traoré, R., Caselles-Dupré, H., Lesort, T., Sun, T., Cai, G., Rodríguez, N.D., Filliat, D.: Discorl: Continual reinforcement learning via policy distillation. CoRR abs/1907.05855
Original
1907
Earlier work this paper cites.
Lange, M.D., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: Continual learning: A comparative study on how to defy forgetting in classification tasks (2019), https://arxiv.org/abs/1909.08383
Original
1909
Earlier work this paper cites.
Farajtabar, M., Azizan, N., Mott, A., Li, A.: Orthogonal gradient descent for continual learning (2019), https://arxiv.org/abs/1910.07104
Original
1910
Earlier work this paper cites.
Caccia, L., Belilovsky, E., Caccia, M., Pineau, J.: Online learned continual compression with adaptative quantization module. arXiv preprint arXiv:1911.08019 (2019), https://arxiv.org/abs/1911.08019
Original
1911
Earlier work this paper cites.
French, R.M.: Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences 3
1999
Earlier work this paper cites.
van der Maaten, L., Hinton, G.: Visualizing data using t-sne (2008)
2008
Earlier work this paper cites.
LeCun, Y., Cortes, C.: MNIST handwritten digit database. public (2010), http://yann.lecun.com/exdb/mnist/
2010
Earlier work this paper cites.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python (2011)
2011
Earlier work this paper cites.
Rusu, A.A., Rabinowitz, N.C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., Hadsell, R.: Progressive neural networks. ArXiv e-prints (jun 2016), https://arxiv.org/abs/1606.04671
Original
2016
Earlier work this paper cites.
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proc. of the national academy of sciences (2017), https://www.pnas.org/content/pnas/114/13/3521.full.pdf
2017
Earlier work this paper cites.

Similar

Then

Hou, S., Pan, X., Loy, C.C., Wang, Z., Lin, D.: Learning a unified classifier incrementally via rebalancing. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)
2019
Closest in time.
Lesort, T., Caselles-Dupré, H., Garcia-Ortiz, M., Goudou, J.F., Filliat, D.: Generative models from the perspective of continual learning. In: IJCNN - International Joint Conference on Neural Networks. Budapest, Hungary (Jul 2019), https://hal.archives-ouvertes.fr/hal-01951954
2019
Closest in time.
Lesort, T., Gepperth, A., Stoian, A., Filliat, D.: Marginal replay vs conditional replay for continual learning. In: International Conference on Artificial Neural Networks. pp. 466–480. Springer (2019), https://arxiv.org/abs/1810.12069
Original
2019
Closest in time.
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., Díaz-Rodríguez, N.: Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges. Information Fusion 58
2019

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 3987–3995. PMLR, International Convention Centre, Sydney, Australia (06–11 Aug 2017), http://proceedings.mlr.press/v70/zenke17a.html

2017

Cited alongside, same era.

Belouadah, E., Popescu, A.: Deesil: Deep-shallow incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 0–0 (2018)

2018

Cited alongside, same era.

Clanuwat, T., Bober-Irizar, M., Kitamoto, A., Lamb, A., Yamamoto, K., Ha, D.: Deep learning for classical japanese literature. CoRR (2018)

2018

Cited alongside, same era.

Ritter, H., Botev, A., Barber, D.: Online structured laplace approximations for overcoming catastrophic forgetting. In: Advances in Neural Information Processing Systems. pp. 3738–3748 (2018)

2018

Cited alongside, same era.

Schwarz, J., Luketina, J., Czarnecki, W.M., Grabska-Barwinska, A., Teh, Y.W., Pascanu, R., Hadsell, R.: Progress & compress: A scalable framework for continual learning. In: ICML (2018), https://arxiv.org/abs/1805.06370

Original

2018

Cited alongside, same era.

Wu, C., Herranz, L., Liu, X., wang, y., van de Weijer, J., Raducanu, B.: Memory replay gans: Learning to generate new categories without forgetting. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31, pp. 5962–5972. Curran Associates, Inc. (2018), http://papers.nips.cc/paper/7836-memory-replay-gans-learning-to-generate-new-categories-without-forgetting.pdf

2018

Cited alongside, same era.

Aljundi, R., , L., Belilovsky, E., Caccia, M., Lin, M., Charlin, L., Tuytelaars, T.: Online continual learning with maximal interfered retrieval. In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32, pp. 11849–11860. Curran Associates, Inc. (2019), http://papers.nips.cc/paper/9357-online-continual-learning-with-maximal-interfered-retrieval.pdf

2019

Cited alongside, same era.

Closest in time.

Doan, T., Bennani, M., Mazoure, B., Rabusseau, G., Alquier, P.: A theoretical analysis of catastrophic forgetting through the ntk overlap matrix. arXiv preprint arXiv:2010.04003 (2020)

Original

2020

Closest in time.

Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Proceedings of the IEEE European Conference on Computer Vision (ECCV) (2020), https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123650086.pdf

2020

Closest in time.

Knoblauch, J., Husain, H., Diethe, T.: Optimal continual learning has perfect memory and is np-hard. arXiv preprint arXiv:2006.05188 (2020)

Original

2020

Closest in time.

Prabhu, A., Torr, P.H., Dokania, P.K.: Gdumb: A simple approach that questions our progress in continual learning (2020), http://www.robots.ox.ac.uk/~tvg/publications/2020/gdumb.pdf

2020

Closest in time.

Douillard, A., Lesort, T.: Continuum: Simple management of complex continual learning scenarios (2021), https://arxiv.org/abs/2102.06253

Original

2021

Closest in time.

Ramasesh, V.V., Dyer, E., Raghu, M.: Anatomy of catastrophic forgetting: Hidden representations and task semantics. In: International Conference on Learning Representations (2021), https://openreview.net/forum?id=LhY8QdUGSuw

2021

Closest in time.