2016

Regret Bounds for Lifelong Learning

Alquier, Pierre, Mai, The Tien, Pontil, Massimiliano

Understand

We consider the problem of transfer learning in an online setting.

Different tasks are presented sequentially and processed by a within-task algorithm.
We propose a lifelong learning strategy which refines the underlying data representation used by the within-task algorithm, thereby transferring information from one task to the next.
We show that when the within-task algorithm comes with some regret bound, our strategy inherits this good property.

Is learning the n-th thing any easier than learning the first?
Thrun, S. (1996) · 1996
Earlier work this paper cites.
A bayesian/information theoretic model of learning to learn via multiple task sampling
Baxter, J. (1997) · 1997
Earlier work this paper cites.
Some pac-bayesian theorems
McAllester, D. A. (1998) · 1998
Earlier work this paper cites.
Learning to Learn
Thrun, S. and Pratt, L. (1998) · 1998
Earlier work this paper cites.
Statistical Learning Theory
Vapnik, V. (1998) · 1998
Earlier work this paper cites.
A model of inductive bias learning
Baxter, J. (2000) · 2000
Earlier work this paper cites.
Statistical learning theory and stochastic optimization
Catoni, O. (2004) · 2001
Earlier work this paper cites.
Algorithmic stability and meta-learning
Maurer, A. (2005) · 2005
Earlier work this paper cites.

A randomized online learning algorithm for better variance control
Audibert, J.-Y. (2006) · 2006
Cited alongside, same era.
Prediction, learning, and games
Cesa-Bianchi, N. and Lugosi, G. (2006) · 2006
Cited alongside, same era.
Logarithmic regret algorithms for online convex optimization
Hazan, E., Agarwal, A., and Kale, S. (2007) · 2007
Cited alongside, same era.
Linear algorithms for online multitask classification
Cavallanti, G., Cesa-Bianchi, N., and Gentile, C. (2010) · 2010
Cited alongside, same era.
Prédiction de suites individuelles et cadre statistique classique: étude de quelques liens autour de la régression parcimonieuse et des techniques d’agrégation
Gerchinovitz, S. (2011) · 2011
Cited alongside, same era.
Online learning and online convex optimization
Shalev-Shwartz, S. (2011) · 2011
Cited alongside, same era.
Learning multiple tasks using shared hypotheses
Crammer, K. and Mansour, Y. (2012) · 2012
Cited alongside, same era.
Sparsity regret bounds for individual sequences in online linear regression
Gerchinovitz, S. (2013) · 2013
Cited alongside, same era.

Sparse coding for multitask and transfer learning
Maurer, A., Pontil, M., and Romera-Paredes, B. (2013) · 2013
Later among the works it cites.
Monte Carlo statistical methods
Robert, C. and Casella, G. (2013) · 2013
Later among the works it cites.
Ella: An efficient lifelong learning algorithm
Ruvolo, P. and Eaton, E. (2013) · 2013
Later among the works it cites.
A pac-bayesian bound for lifelong learning
Pentina, A. and Lampert, C. (2014) · 2014
Later among the works it cites.
Efficient representations for lifelong learning and autoencoding
Balcan, M.-F., Blum, A., and Vempala, S. (2015) · 2015
Later among the works it cites.
Multi-task and lifelong learning of kernels
Pentina, A. and Ben-David, S. (2015) · 2015
Later among the works it cites.
The benefit of multitask representation learning
Maurer, A., Pontil, M., and Romera-Paredes, B. (2016) · 2016
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…