Fetching the paper…
Reading the bibliography…
With the advent and recent ubiquity of foundation models, continual learning (CL) has recently shifted from continual training from scratch to the continual adaptation of pretrained models, seeing particular success on rehearsal-free CL benchmarks (RFCL).
The caltech-ucsd birds-200-2011 dataset
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie · 2011
Earlier work this paper cites.
An empirical investigation of catastrophic forgetting in gradient-based neural networks
Ian J Goodfellow, Mehdi Mirza, Da Xiao, David Farley, Aaron Lin, Aaron Courville, and Yoshua Bengio · 2013
Earlier work this paper cites.
3d object representations for fine-grained categorization
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei · 2013
Earlier work this paper cites.
Unsupervised visual representation learning by context prediction
Carl Doersch, Abhinav Gupta, and Alexei A Efros · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Privacy-preserving deep learning
Reza Shokri and Vitaly Shmatikov · 2015
Earlier work this paper cites.
Progressive neural networks
Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell · 2016
Earlier work this paper cites.
Pathnet: Evolution channels gradient descent in super neural networks
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra · 2017
Earlier work this paper cites.
Gans trained by a two time-scale update rule converge to a local nash equilibrium
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter · 2017
Earlier work this paper cites.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell · 2017
Earlier work this paper cites.
Gradient episodic memory for continual learning
David Lopez-Paz and Marc’Aurelio Ranzato · 2017
Earlier work this paper cites.
Automatic differentiation in pytorch
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer · 2017
Earlier work this paper cites.
icarl: Incremental classifier and representation learning
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert · 2017
Earlier work this paper cites.
Continual learning through synaptic intelligence
Friedemann Zenke, Ben Poole, and Surya Ganguli · 2017
Earlier work this paper cites.
Memory aware synapses: Learning what (not) to forget
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars · 2018
Earlier work this paper cites.
Unsupervised representation learning by predicting image rotations
Spyros Gidaris, Praveer Singh, and Nikos Komodakis · 2018
Earlier work this paper cites.
Measuring catastrophic forgetting in neural networks
Ronald Kemker, Marc McClure, Angelina Abitino, Tyler L Hayes, and Christopher Kanan · 2018
Earlier work this paper cites.
Packnet: Adding multiple tasks to a single network by iterative pruning
Arun Mallya and Svetlana Lazebnik · 2018
Earlier work this paper cites.
Piggyback: Adapting a single network to multiple tasks by learning to mask weights
Arun Mallya, Dillon Davis, and Svetlana Lazebnik · 2018
Earlier work this paper cites.
Lifelong learning with dynamically expandable networks
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang · 2018
Earlier work this paper cites.
Gradient based sample selection for online continual learning
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio · 2019
Earlier work this paper cites.
Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz · 2019
Earlier work this paper cites.
Efficient lifelong learning with a-gem
Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny · 2019
Earlier work this paper cites.
Continual learning via neural pruning
Siavash Golkar, Michael Kagan, and Kyunghyun Cho · 2019
Earlier work this paper cites.
Learning a unified classifier incrementally via rebalancing
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin · 2019
Earlier work this paper cites.
Parameter-efficient transfer learning for nlp
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly · 2019
Cited alongside, same era.
Meta-learning with differentiable convex optimization
Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto · 2019
Cited alongside, same era.
Moment matching for multi-source domain adaptation
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang · 2019
Cited alongside, same era.
Three scenarios for continual learning
Gido M van de Ven and Andreas S Tolias · 2019
Cited alongside, same era.
Pytorch image models, 2019
Ross Wightman · 2019
Cited alongside, same era.
Dark experience for general continual learning: a strong, simple baseline
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara · 2020
Training data-efficient image transformers & distillation through attention
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou · 2021
Later among the works it cites.
BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel · 2022
Later among the works it cites.
Class-incremental continual learning into the extended der-verse
Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, and Simone Calderara · 2022
Later among the works it cites.
Caltech 256, 2022
Gregory Griffin, Alex Holub, and Pietro Perona · 2022
Later among the works it cites.
Lora: Low-rank adaptation of large language models
Edward Hu, Yelong Shen, David Alvarez-Melis, Sewon Han, Xiaodong Li, Weizhu Chen, and Suvrit Sra · 2022
Later among the works it cites.
A simple baseline that questions the use of pretrained-models in continual learning
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Shortcut learning in deep neural networks
R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann · 2020
Cited alongside, same era.
Continual learning in natural language processing: A survey
Nan Rosemary Ke, Colin Raffel, Noam Shazeer, and Adam Roberts · 2020
Cited alongside, same era.
AdapterFusion: Non-Destructive Task Composition for Transfer Learning
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych · 2020
Cited alongside, same era.
Gdumb: A simple approach that questions our progress in continual learning
Ameya Prabhu, Philip Torr, and Puneet Dokania · 2020
Cited alongside, same era.
Efficient meta lifelong-learning with limited memory
Zirui Wang and Derek Hoiem · 2020
Cited alongside, same era.
Supermasks in superposition
Mitchell Wortsman, Ali Farhadi, and Mohammad Rastegari · 2020
Cited alongside, same era.
Paul Janson, Wenxuan Zhang, Rahaf Aljundi, and Mohamed Elhoseiny · 2022
Later among the works it cites.
Continual learning with foundation models: An empirical study of latent replay
Oleksiy Ostapenko, Timothee Lesort, Pau Rodriguez, Md Rifat Arefin, Arthur Douillard, Irina Rish, and Laurent Charlin · 2022
Later among the works it cites.
Momentum-based weight interpolation of strong zero-shot models for continual learning
Zafir Stojanovski, Karsten Roth, and Zeynep Akata · 2022
Later among the works it cites.
Benchmarking omni-vision representation through the lens of visual realms
Zhang Yuanhan, Yin Zhenfei, Shao Jing, and Liu Ziwei · 2022
Later among the works it cites.
Introducing language guidance in prompt-based continual learning
Muhammad Gul Zain Ali Khan, Muhammad Ferjad Naeem, Luc Van Gool, Didier Stricker, Federico Tombari, and Muhammad Zeshan Afzal · 2023
Later among the works it cites.
Adalomo: Low-memory optimization with adaptive learning rate
Kai Lv, Hang Yan, Qipeng Guo, Haijun Lv, and Xipeng Qiu · 2023
Later among the works it cites.
Imad Eddine Marouf, Subhankar Roy, Enzo Tartaglione, and Stéphane Lathuilière · 2023
Later among the works it cites.
RanPAC: Random projections and pre-trained models for continual learning
Mark McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel · 2023
Later among the works it cites.
On the effectiveness of layernorm tuning for continual learning in vision transformers
Thomas De Min, Massimiliano Mancini, Karteek Alahari, Xavier Alameda-Pineda, and Elisa Ricci · 2023
Later among the works it cites.
First session adaptation: A strong replay-free baseline for class-incremental learning
Aristeidis Panos, Yuriko Kobe, Daniel Olmeda Reino, Rahaf Aljundi, and Richard E. Turner · 2023
Later among the works it cites.
A unified continual learning framework with general parameter-efficient tuning
Gao Qiankun, Zhao Chen, Sun Yifan, Xi Teng, Zhang Gang, Ghanem Bernard, and Zhang Jian · 2023
Later among the works it cites.
Controlling text-to-image diffusion by orthogonal finetuning
Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Schölkopf · 2023
Later among the works it cites.
Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira · 2023
Later among the works it cites.
Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality
Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, and Jun Zhu · 2023
Later among the works it cites.
Da-Wei Zhou, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu · 2023
Later among the works it cites.
Ether: Efficient finetuning of large-scale models with hyperplane reflections, 2024
Massimo Bini, Karsten Roth, Zeynep Akata, and Anna Khoreva · 2024
Closest in time.
Vera: Vector-based random matrix adaptation
Dawid J. Kopiczko, Tijmen Blankevoort, and Yuki M. Asano · 2024
Closest in time.
Parameter-efficient orthogonal finetuning via butterfly factorization
Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, Yandong Wen, Michael J. Black, Adrian Weller, and Bernhard Schölkopf · 2024
Closest in time.
Fantastic gains and where to find them: On the existence and prospect of general knowledge transfer between any pretrained model
Karsten Roth, Lukas Thede, A. Sophia Koepke, Oriol Vinyals, Olivier J Henaff, and Zeynep Akata · 2024
Closest in time.