Understand
Continual learning aims to enable machine learning models to learn a general solution space for past and future tasks in a sequential manner.
- Conventional models tend to forget the knowledge of previous tasks while learning a new task, a phenomenon known as catastrophic forgetting.
- When using Bayesian models in continual learning, knowledge from previous tasks can be retained in two ways: 1).
- posterior distributions over the parameters, containing the knowledge gained from inference in previous tasks, which then serve as the priors for the following task; 2).
Built on
Stochastic variational inference
Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley · 2013
Earlier work this paper cites.
Revisiting natural gradient for deep networks
Razvan Pascanu and Yoshua Bengio · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Lei Ba · 2014
Earlier work this paper cites.
Coresets for scalable Bayesian logistic regression
Jonathan Huggins, Trevor Campbell, and Tamara Broderick · 2016
Earlier work this paper cites.
Similar
Stein variational gradient descent: A general purpose Bayesian inference algorithm
Qiang Liu and Dilin Wang · 2016
Cited alongside, same era.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al · 2017
Cited alongside, same era.
Continual learning through synaptic intelligence
Friedemann Zenke, Ben Poole, and Surya Ganguli · 2017
Cited alongside, same era.
Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms
Han Xiao, Kashif Rasul, and Roland Vollgraf · 2017
Cited alongside, same era.
Then
Bayesian compression for deep learning
Christos Louizos, Karen Ullrich, and Max Welling · 2017
Later among the works it cites.
Variational continual learning
Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner · 2018
Later among the works it cites.
Bayesian coreset construction via greedy iterative geodesic ascent
Trevor Campbell and Tamara Broderick · 2018
Later among the works it cites.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…