Continual Learning for Recurrent Neural Networks: an Empirical Evaluation

Understand

Learning continuously during all model lifetime is fundamental to deploy machine learning solutions robust to drifts in the data distribution.

• Advances in Continual Learning (CL) with recurrent neural networks could pave the way to a large number of applications where incoming data is non stationary, like natural language processing and robotics.
• However, the existing body of work on the topic is still fragmented, with approaches which are application-specific and whose assessment is based on heterogeneous learning protocols and datasets.
• In this paper, we organize the literature on CL for sequential data processing by providing a categorization of the contributions and a review of the benchmarks.