Fetching the paper…

Reading the bibliography…

Still no free lunches: the price to pay for tighter PAC-Bayes bounds · Around

2019

Still no free lunches: the price to pay for tighter PAC-Bayes bounds

Guedj, Benjamin, Pujol, Louis

Original
PDF
Post on X

Understand

"No free lunch" results state the impossibility of obtaining meaningful bounds on the error of a learning algorithm without prior assumptions and modelling.

Some models are expensive (strong assumptions, such as as subgaussian tails), others are cheap (simply finite variance).
As it is well known, the more you pay, the more you get: in other words, the most expensive models yield the more interesting bounds.
Recent advances in robust statistics have investigated procedures to obtain tight bounds while keeping the cost minimal.

Built on

I-divergence geometry of probability distributions and minimization problems
Imre Csiszár · 1975
Earlier work this paper cites.
A probabilistic theory of pattern recognition , volume 31
Luc Devroye, László Györfi, and Gábor Lugosi · 1996
Earlier work this paper cites.
Weak convergence
Aad W. van der Vaart and Jon A. Wellner · 1996
Earlier work this paper cites.
Information theory and statistics: A tutorial
Imre Csiszár and Paul C. Shields · 2004
Earlier work this paper cites.

Similar

PAC-Bayesian supervised classification: the thermodynamics of statistical learning
Olivier Catoni · 2007
Cited alongside, same era.
Challenging the empirical mean and empirical variance: a deviation study
Olivier Catoni · 2012
Cited alongside, same era.
Concentration inequalities: A nonasymptotic theory of independence
Stéphane Boucheron, Gábor Lugosi, and Pascal Massart · 2013
Cited alongside, same era.
PAC-Bayesian bounds based on the Rényi divergence
Luc Bégin, Pascal Germain, François Laviolette, and Jean-Francis Roy · 2016
Cited alongside, same era.

Then

Sub-gaussian mean estimators
Luc Devroye, Matthieu Lerasle, Gábor Lugosi, and Roberto I. Oliveira · 2016
Later among the works it cites.
Simpler PAC-Bayesian bounds for hostile data
Pierre Alquier and Benjamin Guedj · 2018
Later among the works it cites.
A primer on PAC-Bayesian learning
Original
Benjamin Guedj · 2019
Closest in time.
Lecture notes: Selected topics on robust statistical learning theory
Original
Matthieu Lerasle · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…