Fetching the paper…
Reading the bibliography…
Transformer neural networks are increasingly replacing prior architectures in a wide range of applications in different data modalities.
Catastrophic forgetting, rehearsal and pseudorehearsal
Anthony Robins · 1995
Earlier work this paper cites.
Multitask learning
Rich Caruana · 1998
Earlier work this paper cites.
Catastrophic forgetting in connectionist networks
Robert M French · 1999
Earlier work this paper cites.
Model compression
Cristian Buciluundefined, Rich Caruana, and Alexandru Niculescu-Mizil · 2006
Earlier work this paper cites.
Lifelong machine learning systems: Beyond learning algorithms
Daniel L Silver, Qiang Yang, and Lianghao Li · 2013
Earlier work this paper cites.
Microsoft COCO: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Exploring models and data for image question answering, 2015
Mengye Ren, Ryan Kiros, and Richard Zemel · 2015
Earlier work this paper cites.
Fitnets: Hints for thin deep nets, 2015
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio · 2015
Earlier work this paper cites.
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver · 2015
Earlier work this paper cites.
Using task features for zero-shot knowledge transfer in lifelong learning
David Isele, Mohammad Rostami, and Eric Eaton · 2016
Earlier work this paper cites.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell · 2016
Earlier work this paper cites.
Learning efficient object detection models with knowledge distillation
Guobin Chen, Wongun Choi, Xiang Yu, Tony X. Han, and Manmohan Chandraker · 2017
Earlier work this paper cites.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al · 2017
Earlier work this paper cites.
Multi-agent distributed lifelong learning for collective knowledge acquisition
Mohammad Rostami, Soheil Kolouri, Kyungnam Kim, and Eric Eaton · 2017
Earlier work this paper cites.
Continual learning with deep generative replay
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Attention is all you need, 2017
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
A deeper look at experience replay
Shangtong Zhang and Richard S Sutton · 2017
Earlier work this paper cites.
Places: A 10 million image database for scene recognition
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba · 2017
Earlier work this paper cites.
Memory aware synapses: Learning what (not) to forget
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars · 2018
Earlier work this paper cites.
Lifelong machine learning
Zhiyuan Chen and Bing Liu · 2018
Earlier work this paper cites.
The inaturalist species classification and detection dataset, 2018
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie · 2018
Earlier work this paper cites.
A corpus for reasoning about natural language grounded in photographs, 2018
Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi · 2018
Earlier work this paper cites.
Gqa: A new dataset for real-world visual reasoning and compositional question answering, 2019
Drew A. Hudson and Christopher D. Manning · 2019
Earlier work this paper cites.
A comparative study on transformer vs rnn in speech applications
Shigeki Karita, Nanxin Chen, Tomoki Hayashi, Takaaki Hori, Hirofumi Inaguma, Ziyan Jiang, Masao Someki, Nelson Enrique Yalta Soplin, Ryuichi Yamamoto, Xiaofei Wang, et al · 2019
Earlier work this paper cites.
Ok-vqa: A visual question answering benchmark requiring external knowledge, 2019
Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi · 2019
Earlier work this paper cites.
Experience replay for continual learning
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne · 2019
Earlier work this paper cites.
Complementary learning for overcoming catastrophic forgetting using experience replay
Mohammad Rostami, Soheil Kolouri, and Praveen K Pilly · 2019
Earlier work this paper cites.
Vl-bert: Pre-training of generic visual-linguistic representations
Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai · 2019
Cited alongside, same era.
Videobert: A joint model for video and language representation learning
Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid · 2019
Cited alongside, same era.
CommonsenseQA: A question answering challenge targeting commonsense knowledge
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant · 2019
Cited alongside, same era.
Lxmert: Learning cross-modality encoder representations from transformers
Hao Tan and Mohit Bansal · 2019
Cited alongside, same era.
Three scenarios for continual learning
Gido M Van de Ven and Andreas S Tolias · 2019
Cited alongside, same era.
Contextual attention network: Transformer meets u-net
Reza Azad, Moein Heidari, Yuli Wu, and Dorit Merhof · 2022
Later among the works it cites.
Mult: An end-to-end multitask learning transformer
Deblina Bhattacharjee, Tong Zhang, Sabine Süsstrunk, and Mathieu Salzmann · 2022
Later among the works it cites.
Georgios Chochlakis, Tejas Srinivasan, Jesse Thomason, and Shrikanth Narayanan · 2022
Later among the works it cites.
Dytox: Transformers for continual learning with dynamic token expansion
Arthur Douillard, Alexandre Ramé, Guillaume Couairon, and Matthieu Cord · 2022
Later among the works it cites.
Continual learning with transformers for image classification
Beyza Ermis, Giovanni Zappella, Martin Wistuba, Aditya Rawal, and Cédric Archambeau · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Johannes Von Oswald, Christian Henning, João Sacramento, and Benjamin F Grewe · 2019
Cited alongside, same era.
Visual entailment: A novel task for fine-grained image understanding, 2019
Ning Xie, Farley Lai, Derek Doran, and Asim Kadav · 2019
Cited alongside, same era.
HellaSwag: Can a machine really finish your sentence?
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi · 2019
Cited alongside, same era.
PIQA: Reasoning about physical commonsense in natural language
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al · 2020
Cited alongside, same era.
Uniter: Universal image-text representation learning
Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu · 2020
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2020
Cited alongside, same era.
Revisiting fundamentals of experience replay
William Fedus, Prajit Ramachandran, Rishabh Agarwal, Yoshua Bengio, Hugo Larochelle, Mark Rowland, and Will Dabney · 2020
Cited alongside, same era.
Memory efficient continual learning with transformers
Beyza Ermis, Giovanni Zappella, Martin Wistuba, Aditya Rawal, and Cedric Archambeau · 2022
Later among the works it cites.
A survey on vision transformer
Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, et al · 2022
Later among the works it cites.
Gradient episodic memory for continual learning, 2022
David Lopez-Paz and Marc’Aurelio Ranzato · 2022
Later among the works it cites.
Towards exemplar-free continual learning in vision transformers: an account of attention, functional and weight regularization
Francesco Pelosin, Saurav Jha, Andrea Torsello, Bogdan Raducanu, and Joost van de Weijer · 2022
Later among the works it cites.
Learning audio-visual speech representation by masked multimodal cluster prediction, 2022
Bowen Shi, Wei-Ning Hsu, Kushal Lakhotia, and Abdelrahman Mohamed · 2022
Later among the works it cites.
Continual learning with lifelong vision transformer
Zhen Wang, Liu Liu, Yiqun Duan, Yajing Kong, and Dacheng Tao · 2022
Later among the works it cites.
Deepchange: A large long-term person re-identification benchmark with clothes change, 2022
Peng Xu and Xiatian Zhu · 2022
Later among the works it cites.
Continual learning for natural language generations with transformer calibration
Peng Yang, Dingcheng Li, and Ping Li · 2022
Later among the works it cites.
A survey on negative transfer
Wen Zhang, Lingfei Deng, Lei Zhang, and Dongrui Wu · 2022
Later among the works it cites.
Vision transformers in image restoration: A survey
Anas M Ali, Bilel Benjdira, Anis Koubaa, Walid El-Shafai, Zahid Khan, and Wadii Boulila · 2023
Later among the works it cites.
Task-attentive transformer architecture for continual learning of vision-and-language tasks using knowledge distillation
Yuliang Cai, Jesse Thomason, and Mohammad Rostami · 2023
Later among the works it cites.
Self-paced weight consolidation for continual learning
Wei Cong, Yang Cong, Gan Sun, Yuyang Liu, and Jiahua Dong · 2023
Later among the works it cites.
Online aware synapse weighted autoencoder for recovering random missing data in wastewater treatment process
Honggui Han, Meiting Sun, and Fangyu Li · 2023
Later among the works it cites.
Class-incremental learning using generative experience replay based on time-aware regularization
Zizhao Hu and Mohammad Rostami · 2023
Later among the works it cites.
Lvit: language meets vision transformer in medical image segmentation
Zihan Li, Yunxiang Li, Qingde Li, Puyang Wang, Dazhou Guo, Le Lu, Dakai Jin, You Zhang, and Qingqi Hong · 2023
Later among the works it cites.
Continual detection transformer for incremental object detection
Yaoyao Liu, Bernt Schiele, Andrea Vedaldi, and Christian Rupprecht · 2023
Later among the works it cites.
History repeats: Overcoming catastrophic forgetting for event-centric temporal knowledge graph completion
Mehrnoosh Mirtaheri, Mohammad Rostami, and Aram Galstyan · 2023
Later among the works it cites.
D3former: Debiased dual distilled transformer for incremental learning
Abdelrahman Mohamed, Rushali Grandhe, KJ Joseph, Salman Khan, and Fahad Khan · 2023
Later among the works it cites.
Cognitively inspired learning of incremental drifting concepts
Mohammad Rostami and Aram Galstyan · 2023
Later among the works it cites.
Overcoming concept shift in domain-aware settings through consolidated internal distributions
Mohammad Rostami and Aram Galstyan · 2023
Later among the works it cites.
I2i: Initializing adapters with improvised knowledge
Tejas Srinivasan, Furong Jia, Mohammad Rostami, and Jesse Thomason · 2023
Later among the works it cites.
Continual few-shot learning with transformer adaptation and knowledge regularization
Xin Wang, Yue Liu, Jiapei Fan, Weigao Wen, Hui Xue, and Wenwu Zhu · 2023
Later among the works it cites.
Multimodal learning with transformers: A survey
Peng Xu, Xiatian Zhu, and David A Clifton · 2023
Later among the works it cites.
Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond
Qiming Zhang, Yufei Xu, Jing Zhang, and Dacheng Tao · 2023
Later among the works it cites.