Fetching the paper…
Reading the bibliography…
While a diverse collection of continual learning (CL) methods has been proposed to prevent catastrophic forgetting, a thorough investigation of their effectiveness for processing sequential data with recurrent neural networks (RNNs) is lacking.
Using Pseudo-Recurrent Connectionist Networks to Solve the Problem of Sequential Learning
Robert M. French · 1970
Earlier work this paper cites.
Finding structure in time
Jeffrey L. Elman · 1990
Earlier work this paper cites.
Semi-distributed Representations and Catastrophic Forgetting in Connectionist Networks
Robert M. French · 1992
Earlier work this paper cites.
A practical bayesian framework for backpropagation networks
David JC MacKay · 1992
Earlier work this paper cites.
Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks
Jürgen Schmidhuber · 1992
Earlier work this paper cites.
A ‘self-referential’ weight matrix
Jürgen Schmidhuber · 1993
Earlier work this paper cites.
Dynamically Constraining Connectionist Networks to Produce Distributed, Orthogonal Representations to Reduce Catastrophic Interference
Robert M. French · 1994
Earlier work this paper cites.
Kernel principal component analysis
Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller · 1997
Earlier work this paper cites.
Bidirectional recurrent neural networks
M. Schuster and K. K. Paliwal · 1997
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Self-refreshing memory in artificial neural networks: learning temporal sequences without catastrophic forgetting
Bernard Ans, Stéphane Rousset, Robert M. French, and Serban Musca · 2004
Earlier work this paper cites.
Consciousness CLEARS the mind
Stephen Grossberg · 2007
Earlier work this paper cites.
Mitigation of catastrophic interference in neural networks using a fixed expansion layer
Robert Coop and Itamar Arel · 2012
Earlier work this paper cites.
Mitigation of catastrophic forgetting in recurrent neural networks using a fixed expansion layer
Robert Coop and Itamar Arel · 2013
Earlier work this paper cites.
Predicting parameters in deep learning
Misha Denil, Babak Shakibi, Laurent Dinh, Marc’Aurelio Ranzato, and Nando De Freitas · 2013
Earlier work this paper cites.
The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects
Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin · 2013
Earlier work this paper cites.
What regularized auto-encoders learn from the data-generating distribution
Guillaume Alain and Yoshua Bengio · 2014
Earlier work this paper cites.
Learning stochastic recurrent networks
Justin Bayer and Christian Osendorfer · 2014
Earlier work this paper cites.
Learning phrase representations using rnn encoder-decoder for statistical machine translation
Kyunghyun Cho, B van Merrienboer, Caglar Gulcehre, F Bougares, H Schwenk, and Yoshua Bengio · 2014
Earlier work this paper cites.
Alex Graves, Greg Wayne, and Ivo Danihelka · 2014
Earlier work this paper cites.
Auto-encoding variational bayes
Diederik P. Kingma and Max Welling · 2014
Earlier work this paper cites.
Stochastic backpropagation and approximate inference in deep generative models
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra · 2014
Earlier work this paper cites.
Learning to execute, 2014
Wojciech Zaremba and Ilya Sutskever · 2014
Earlier work this paper cites.
A recurrent latent variable model for sequential data
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio · 2015
Earlier work this paper cites.
Learning both weights and connections for efficient neural network
Song Han, Jeff Pool, John Tran, and William Dally · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Net2Net: Accelerating Learning via Knowledge Transfer
Tianqi Chen, Ian Goodfellow, and Jonathon Shlens · 2016
Cited alongside, same era.
Incremental Sequence Learning
Edwin D. de Jong · 2016
Cited alongside, same era.
Universal dependencies v1: A multilingual treebank collection
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman · 2016
Cited alongside, same era.
Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss
Barbara Plank, Anders Søgaard, and Yoav Goldberg · 2016
Cited alongside, same era.
Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell · 2016
Cited alongside, same era.
Continuous learning in a hierarchical multiscale neural network
Thomas Wolf, Julien Chaumond, and Clement Delangue · 2018
Later among the works it cites.
Lifelong learning with dynamically expandable networks
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang · 2018
Later among the works it cites.
Entropy-SGD: Biasing gradient descent into wide valleys
Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina · 2019
Later among the works it cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Later among the works it cites.
Task agnostic continual learning via meta learning
Xu He, Jakub Sygnowski, Alexandre Galashov, Andrei A Rusu, Yee Whye Teh, and Razvan Pascanu · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Expert gate: Lifelong learning with a network of experts
Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars · 2017
Cited alongside, same era.
Audio set: An ontology and human-labeled dataset for audio events
Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal, and Marvin Ritter · 2017
Cited alongside, same era.
Memory augmented neural networks with wormhole connections
Caglar Gulcehre, Sarath Chandar, and Yoshua Bengio · 2017
Cited alongside, same era.
Hypernetworks
David Ha, Andrew Dai, and Quoc Le · 2017
Cited alongside, same era.
Toward Continual Learning for Conversational Agents
Sungjin Lee · 2017
Cited alongside, same era.
Learning without forgetting
Zhizhong Li and Derek Hoiem · 2017
Cited alongside, same era.
icarl: Incremental classifier and representation learning
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert · 2017
Cited alongside, same era.
Sequence tagging with contextual and non-contextual subword representations: A multilingual evaluation
Benjamin Heinzerling and Michael Strube · 2019
Later among the works it cites.
Sentiment classification by leveraging the shared knowledge from a sequence of domains
Guangyi Lv, Shuai Wang, Bing Liu, Enhong Chen, and Kun Zhang · 2019
Later among the works it cites.
Continual lifelong learning with neural networks: A review
German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter · 2019
Later among the works it cites.
Making good on lstms unfulfilled promise
Daniel Philps, Artur d’Avila Garcez, and Tillman Weyde · 2019
Later among the works it cites.
Language models are unsupervised multitask learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever · 2019
Later among the works it cites.
A Study on Catastrophic Forgetting in Deep LSTM Networks
Monika Schak and Alexander Gepperth · 2019
Later among the works it cites.
Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift
Jasper Snoek, Yaniv Ovadia, Emily Fertig, Balaji Lakshminarayanan, Sebastian Nowozin, D Sculley, Joshua Dillon, Jie Ren, and Zachary Nado · 2019
Later among the works it cites.
Overcoming Catastrophic Forgetting During Domain Adaptation of Neural Machine Translation
Brian Thompson, Jeremy Gwinnup, Huda Khayrallah, Kevin Duh, and Philipp Koehn · 2019
Later among the works it cites.
Three scenarios for continual learning
Gido M. van de Ven and Andreas S. Tolias · 2019
Later among the works it cites.
Task representations in neural networks trained to perform many cognitive tasks
Guangyu Robert Yang, Madhura R Joglekar, H Francis Song, William T Newsome, and Xiao-Jing Wang · 2019
Later among the works it cites.
Progressive memory banks for incremental domain adaptation
Nabiha Asghar, Lili Mou, Kira A. Selby, Kevin D. Pantasdo, Pascal Poupart, and Xin Jiang · 2020
Closest in time.
Continual Learning with Gated Incremental Memories for sequential data processing
Andrea Cossu, Antonio Carta, and Davide Bacciu · 2020
Closest in time.
Organizing recurrent network dynamics by task-computation to enable continual learning
Lea Duncker, Laura Driscoll, Krishna V Shenoy, Maneesh Sahani, and David Sussillo · 2020
Closest in time.
Class-agnostic continual learning of alternating languages and domains
Germán Kruszewski, Ionut-Teodor Sorodoc, and Tomas Mikolov · 2020
Closest in time.
Compositional language continual learning
Yuanpeng Li, Liang Zhao, Kenneth Church, and Mohamed Elhoseiny · 2020
Closest in time.
Sequential domain adaptation through elastic weight consolidation for sentiment analysis
Avinash Madasu and Vijjini Anvesh Rao · 2020
Closest in time.
Continual learning of recurrent neural networks by locally aligning distributed representations
Alexander Ororbia, Ankur Mali, C Lee Giles, and Daniel Kifer · 2020
Closest in time.
Toward training recurrent neural networks for lifelong learning
Shagun Sodhani, Sarath Chandar, and Yoshua Bengio · 2020
Closest in time.
A modeling framework for adaptive lifelong learning with transfer and savings through gating in the prefrontal cortex
Ben Tsuda, Kay M. Tye, Hava T. Siegelmann, and Terrence J. Sejnowski · 2020
Closest in time.
Continual learning with hypernetworks
Johannes von Oswald, Christian Henning, João Sacramento, and Benjamin F. Grewe · 2020
Closest in time.