Fetching the paper…
Reading the bibliography…
Existing continual learning (CL) methods mainly rely on fine-tuning or adapting large language models (LLMs).
Three scenarios for continual learning
Gido M Van de Ven and Andreas S Tolias. 2019 · 1904
Earlier work this paper cites.
Catastrophic interference in connectionist networks: The sequential learning problem
Michael McCloskey and Neal J Cohen. 1989 · 1989
Earlier work this paper cites.
The mahalanobis distance
Roy De Maesschalck, Delphine Jouan-Rimbaud, and Désiré L Massart. 2000 · 2000
Earlier work this paper cites.
Dbpedia: A nucleus for a web of open data
Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007 · 2007
Earlier work this paper cites.
icarl: Incremental classifier and representation learning
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017 · 2010
Earlier work this paper cites.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017 · 2017
Earlier work this paper cites.
Continual learning with deep generative replay
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017 · 2017
Earlier work this paper cites.
Lifelong machine learning
Zhiyuan Chen and Bing Liu. 2018 · 2018
Earlier work this paper cites.
Overcoming catastrophic interference using conceptor-aided backpropagation
Xu He and Herbert Jaeger. 2018 · 2018
Earlier work this paper cites.
Fundamentals of probability and stochastic processes with applications to communications
Kun Il Park, M Park, et al. 2018 · 2018
Earlier work this paper cites.
Overcoming catastrophic forgetting with hard attention to the task
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. 2018 · 2018
Earlier work this paper cites.
Psycholinguistics meets continual learning: Measuring catastrophic forgetting in visual question answering
Claudio Greco, Barbara Plank, Raquel Fernández, and Raffaella Bernardi. 2019 · 2019
Earlier work this paper cites.
An evaluation dataset for intent classification and out-of-scope prediction
Stefan Larson, Anish Mahendran, Joseph J Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K Kummerfeld, Kevin Leach, Michael A Laurenzano, Lingjia Tang, et al. 2019 · 2019
Earlier work this paper cites.
Compositional language continual learning
Yuanpeng Li, Liang Zhao, Kenneth Church, and Mohamed Elhoseiny. 2019 · 2019
Earlier work this paper cites.
Continual learning for sentence representations using conceptors
Tianlin Liu, Lyle Ungar, and João Sedoc. 2019 · 2019
Earlier work this paper cites.
Sentence-bert: Sentence embeddings using siamese bert-networks
Nils Reimers and Iryna Gurevych. 2019 · 2019
Earlier work this paper cites.
A progressive model to enable continual learning for semantic slot filling
Yilin Shen, Xiangyu Zeng, and Hongxia Jin. 2019 · 2019
Earlier work this paper cites.
Conditional channel gated networks for task-aware continual learning
Davide Abati, Jakub Tomczak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehteshami Bejnordi. 2020 · 2020
Earlier work this paper cites.
Efficient intent detection with dual sentence encoders
Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. 2020 · 2020
Earlier work this paper cites.
Lifelong language knowledge distillation
Yung-Sung Chuang, Shang-Yu Su, and Yun-Nung Chen. 2020 · 2020
Cited alongside, same era.
Neural topic modeling with continual lifelong learning
Pankaj Gupta, Yatin Chaudhary, Thomas Runkler, and Hinrich Schuetze. 2020 · 2020
Cited alongside, same era.
itaml: An incremental task-agnostic meta-learning approach
Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Mubarak Shah. 2020 · 2020
Cited alongside, same era.
Lamol: Language modeling for lifelong language learning
Fan-Keng Sun, Cheng-Hao Ho, and Hung-Yi Lee. 2020 · 2020
Cited alongside, same era.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020 · 2020
Cited alongside, same era.
A continual learning survey: Defying forgetting in classification tasks
Continual prompt tuning for dialog state tracking
Qi Zhu, Bing Li, Fei Mi, Xiaoyan Zhu, and Minlie Huang. 2022 · 2022
Later among the works it cites.
Reasoning with language model is planning with world model
Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. 2023 · 2023
Later among the works it cites.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 · 2023
Later among the works it cites.
Continual pre-training of language models
Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. 2023 · 2023
Later among the works it cites.
Learnability and algorithm for continual learning
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, and Bing Liu. 2023 · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. 2021 · 2021
Cited alongside, same era.
Continual learning for task-oriented dialogue system with iterative network pruning, expanding and masking
Binzong Geng, Fajie Yuan, Qiancheng Xu, Ying Shen, Ruifeng Xu, and Min Yang. 2021 · 2021
Cited alongside, same era.
Lora: Low-rank adaptation of large language models
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2021 · 2021
Cited alongside, same era.
Continual learning for text classification with information disentanglement based regularization
Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, and Diyi Yang. 2021 · 2021
Cited alongside, same era.
Adapting bert for continual learning of a sequence of aspect sentiment classification tasks
Zixuan Ke, Hu Xu, and Bing Liu. 2021b · 2021
Cited alongside, same era.
Der: Dynamically expandable representation for class incremental learning
Shipeng Yan, Jiangwei Xie, and Xuming He. 2021 · 2021
Cited alongside, same era.
Llm. int8 (): 8-bit matrix multiplication for transformers at scale, 2022
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022 · 2022
Cited alongside, same era.
Tatsuya Konishi, Mori Kurokawa, Chihiro Ono, Zixuan Ke, Gyuhak Kim, and Bing Liu. 2023 · 2023
Later among the works it cites.
Continual learning for multilingual neural machine translation via dual importance-based model division
Junpeng Liu, Kaiyu Huang, Hao Yu, Jiuyi Li, Jinsong Su, and Degen Huang. 2023 · 2023
Later among the works it cites.
Lifelong sequence generation with dynamic module expansion and adaptation
Chengwei Qin, Chen Chen, and Shafiq Joty. 2023 · 2023
Later among the works it cites.
Progressive prompts: Continual learning for language models
Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi. 2023 · 2023
Later among the works it cites.
Class-incremental learning based on label generation
Yijia Shao, Yiduo Guo, Dongyan Zhao, and Bing Liu. 2023 · 2023
Later among the works it cites.
Longlora: Efficient fine-tuning of long-context large language models
Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2024 · 2024
Closest in time.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024 · 2024
Closest in time.
Continual learning with semi-supervised contrastive distillation for incremental neural machine translation
Yunlong Liang, Fandong Meng, Jiaan Wang, Jinan Xu, Yufeng Chen, and Jie Zhou. 2024 · 2024
Closest in time.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. 2024 · 2024
Closest in time.
Focused transformer: Contrastive training for context scaling
Szymon Tworkowski, Konrad Staniszewski, Mikołaj Pacek, Yuhuai Wu, Henryk Michalewski, and Piotr Miłoś. 2024 · 2024
Closest in time.
Rehearsal-free modular and compositional continual learning for language models
Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, and Hinrich Schütze. 2024b · 2024
Closest in time.
Inscl: A data-efficient continual learning paradigm for fine-tuning large language models with instructions
Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. 2024c · 2024
Closest in time.
Tree of thoughts: Deliberate problem solving with large language models
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024 · 2024
Closest in time.