Fetching the paper…
Reading the bibliography…
We propose a causal framework to explain the catastrophic forgetting in Class-Incremental Learning (CIL) and then derive a novel distillation method that is orthogonal to the existing anti-forgetting techniques, such as data replay and feature/label distillation.
Catastrophic forgetting, rehearsal and pseudorehearsal
A. Robins · 1995
Earlier work this paper cites.
Lifelong Learning Algorithms
Sebastian Thrun · 1998
Earlier work this paper cites.
An analysis of the exponentiated gradient descent algorithm
S. I. Hill and R. C. Williamson · 1999
Earlier work this paper cites.
On the momentum term in gradient descent learning algorithms
Ning Qian · 1999
Earlier work this paper cites.
Causality: Models, reasoning, and inference, second edition
Judea Pearl · 2000
Earlier work this paper cites.
Incremental and Decremental Support Vector Machine Learning
Gert Cauwenberghs and Tomaso Poggio · 2001
Earlier work this paper cites.
Catastrophic Forgetting in Connectionist Networks
Robert French · 2006
Earlier work this paper cites.
ImageNet: a Large-Scale Hierarchical Image Database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei Fei Li · 2009
Earlier work this paper cites.
Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky · 2012
Earlier work this paper cites.
From N to N+1: Multiclass Transfer Incremental Learning
Ilja Kuzborskij, Francesco Orabona, and Barbara Caputo · 2013
Earlier work this paper cites.
On the importance of initialization and momentum in deep learning
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton · 2013
Earlier work this paper cites.
Interpretation and Identification of Causal Mediation
Judea Pearl · 2014
Earlier work this paper cites.
Visual Causal Feature Learning
Krzysztof Chalupka, Pietro Perona, and Frederick Eberhardt · 2015
Earlier work this paper cites.
Distilling the Knowledge in a Neural Network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Purkinje cell activity during classical conditioning with different conditional stimuli explains central tenet of Rescorla-Wagner model
Anders Rasmussen, Riccardo Zucca, Fredrik Johansson, Dan-Anders Jirenhed, and Germund Hesslow · 2015
Earlier work this paper cites.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell · 2016
Earlier work this paper cites.
Causal inference in statistics: A primer
Judea Pearl, Madelyn Glymour, and Nicholas P Jewell · 2016
Earlier work this paper cites.
The Biology of Forgetting–A Perspective
Ronald Davis and Yi Zhong · 2017
Cited alongside, same era.
Deep Generative Dual Memory Network for Continual Learning
Nitin Kamra, Umang Gupta, and Yan Liu · 2017
Cited alongside, same era.
Learning without Forgetting
Zhizhong Li and Derek Hoiem · 2017
Cited alongside, same era.
iCaRL: Incremental Classifier and Representation Learning
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert · 2017
Cited alongside, same era.
Continual Learning with Deep Generative Replay
Hanul Shin, Jung Lee, Jaehong Kim, and Jiwon Kim · 2017
Cited alongside, same era.
End-to-End Incremental Learning
Francisco M. Castro, Manuel J. Marin-Jimenez, Nicolas Guil, Cordelia Schmid, and Karteek Alahari · 2018
Cited alongside, same era.
Causal Intervention for Weakly Supervised Semantic Segmentation
Zhang Dong, Zhang Hanwang, Tang Jinhui, Hua Xiansheng, and Sun Qianru · 2020
Later among the works it cites.
PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning
Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle · 2020
Later among the works it cites.
Improving Confidence Estimates for Unfamiliar Examples
Zhizhong Li and Derek Hoiem · 2020
Later among the works it cites.
Mnemonics Training: Multi-Class Incremental Learning Without Forgetting
Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, and Qianru Sun · 2020
Later among the works it cites.
Counterfactual vqa: A cause-effect look at language bias
Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr · 2018
Cited alongside, same era.
Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
Yen-Chang Hsu, Yen-Cheng Liu, and Zsolt Kira · 2018
Cited alongside, same era.
FearNet: Brain-Inspired Model for Incremental Learning
Ronald Kemker and Christopher Kanan · 2018
Cited alongside, same era.
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
Kimin Lee, Kibok Lee, H. Lee, and Jinwoo Shin · 2018
Cited alongside, same era.
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
Arun Mallya and Svetlana Lazebnik · 2018
Cited alongside, same era.
Continual Lifelong Learning with Neural Networks: A Review
German Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Stefan Wermter · 2018
Cited alongside, same era.
Jiaxin Qi, Yulei Niu, Jianqiang Huang, and Hanwang Zhang · 2020
Later among the works it cites.
Long-tailed classification by keeping the good and removing the bad momentum causal effect
Kaihua Tang, Jianqiang Huang, and Hanwang Zhang · 2020
Later among the works it cites.
Unbiased scene graph generation from biased training
Kaihua Tang, Yulei Niu, Jianqiang Huang, Jiaxin Shi, and Hanwang Zhang · 2020
Later among the works it cites.
Topology-Preserving Class-Incremental Learning
Xiaoyu Tao, Xinyuan Chang, Xiaopeng Hong, Xing Wei, and Yihong Gong · 2020
Later among the works it cites.
Few-Shot Class-Incremental Learning
Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong · 2020
Later among the works it cites.
Brain-inspired replay for continual learning with artificial neural networks
Gido van de Ven, Hava Siegelmann, and Andreas Tolias · 2020
Later among the works it cites.
Visual commonsense r-cnn
Tan Wang, Jianqiang Huang, Hanwang Zhang, and Qianru Sun · 2020
Later among the works it cites.
Deconfounded image captioning: A causal retrospect
Xu Yang, Hanwang Zhang, and Jianfei Cai · 2020
Later among the works it cites.
Interventional few-shot learning
Zhongqi Yue, Hanwang Zhang, Qianru Sun, and Xian-Sheng Hua · 2020
Later among the works it cites.
Adaptive Aggregation Networks for Class-Incremental Learning
Yaoyao Liu, Bernt Schiele, and Qianru Sun · 2021
Closest in time.
Few-Shot Incremental Learning with Continually Evolved Classifiers
Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, and Yinghui Xu · 2021
Closest in time.