2019

Safe-Bayesian Generalized Linear Regression

de Heide, Rianne, Kirichenko, Alisa, Mehta, Nishant et al.

Understand

We study generalized Bayesian inference under misspecification, i.e.

when the model is 'wrong but useful'.
Generalized Bayes equips the likelihood with a learning rate $\eta$.
We show that for generalized linear models (GLMs), $\eta$-generalized Bayes concentrates around the best approximation of the truth within the model for specific $\eta \neq 1$, even under severely misspecified noise, as long as the tails of the true distribution are exponential.

Built on

Information and Exponential Families in Statistical Theory
O. E. Barndorff-Nielsen · 1978
Earlier work this paper cites.
Generalized Linear Models
P. McCullagh and J. Nelder · 1989
Earlier work this paper cites.
Minimum complexity density estimation
A. R. Barron and T. M. Cover · 1991
Earlier work this paper cites.
Regression shrinkage and selection via the lasso
R. Tibshirani · 1996
Earlier work this paper cites.
A decision-theoretic extension of stochastic complexity and its applications to learning
K. Yamanishi · 1998
Earlier work this paper cites.
Estimation of mixture models
Q. J. Li · 1999
Earlier work this paper cites.
On Bayesian consistency
S. Walker and N. L. Hjort · 2002
Earlier work this paper cites.
When ignorance is bliss
P. D. Grünwald and J. Y. Halpern · 2004
Earlier work this paper cites.
Optimal aggregation of classifiers in statistical learning
A. B. Tsybakov · 2004
Earlier work this paper cites.
Local Rademacher complexities
P. L. Bartlett, O. Bousquet, and S. Mendelson · 2005
Earlier work this paper cites.
Prediction, Learning and Games
N. Cesa-Bianchi and G. Lugosi · 2006
Earlier work this paper cites.
Catching up faster in Bayesian model selection and model averaging
T. van. Erven, P. Grünwald, and S. de. Rooij · 2007
Earlier work this paper cites.

Similar

Suboptimal behavior of Bayes and MDL in classification under misspecification
P. Grünwald and J. Langford · 2007
Cited alongside, same era.
The Minimum Description Length Principle
P. D. Grünwald · 2007
Cited alongside, same era.
The Bayesian lasso
T. Park and G. Casella · 2008
Cited alongside, same era.
Predicting quantitative traits with regression models for dense molecular markers and pedigree
G. de los Campos, D. Naya, J. Gianola, J. Crossa, A. Legarra, E. Manfredi, K. Weigel, and J. Cotes · 2009
Cited alongside, same era.
The horseshoe estimator for sparse signals
C. M. Carvalho, N. G. Polson, and J. G. Scott · 2010
Cited alongside, same era.
Openair - an R package for air quality data analysis
D. C. Carslaw and K. Ropkins · 2012
Cited alongside, same era.
The safe Bayesian: learning the learning rate via the mixability gap
P. D. Grünwald · 2012
Cited alongside, same era.
Risk of Bayesian inference in misspecified models, and the sandwich covariance matrix
U. K. Müller · 2013
Cited alongside, same era.
Bayesian inference for logistic models using Pólya–gamma latent variables
N. G. Polson, J. G. Scott, and J. Windle · 2013
Cited alongside, same era.
Package weatherdata, get weather data from the web., 06 2014
R. Narasimhan · 2014
Cited alongside, same era.
Understanding machine learning: From theory to algorithms
S. Shalev-Shwartz and S. Ben-David · 2014
Cited alongside, same era.
From ε \varepsilon -entropy to KL-entropy: Analysis of minimum information complexity density estimation
T. Zhang
Cited in the paper.

Then

Sampling pólya-gamma random variates: alternate and approximate techniques
Original
J. Windle, N. G. Polson, and J. G. Scott · 2014
Later among the works it cites.
The openair manual - open-source tools for analysing air pollution data. Manual for version 1.1-4
D. C. Carslaw · 2015
Later among the works it cites.
Fast rates in statistical and online learning
T. van Erven, P. Grünwald, N. A. Mehta, M. D. Reid, and R. C. Williamson · 2015
Later among the works it cites.
SafeBayes: Generalized and Safe-Bayesian Ridge and Lasso Regression , 2016
R. de Heide · 2016
Later among the works it cites.
horseshoe: Implementation of the Horseshoe prior
S. van der Pas, J. Scott, A. Chakraborty, and A. Bhattacharya · 2016
Later among the works it cites.
Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it
P. Grünwald and T. van Ommen · 2017
Later among the works it cites.
Assigning a value to a power likelihood in a general Bayesian model
C. Holmes and S. Walker · 2017
Later among the works it cites.
Empirical Bayes posterior concentration in sparse high-dimensional linear models
R. Martin, R. Mess, and S. G. Walker · 2017
Later among the works it cites.
Calibrating general posterior credible regions
Original
N. Syring and R. Martin · 2017
Later among the works it cites.
Using stacking to average Bayesian predictive distributions
Y. Yao, A. Vehtari, D. Simpson, and A. Gelman · 2018
Later among the works it cites.
Fast rates for general unbounded loss functions: from ERM to generalized Bayes
Original
P. D. Grünwald and N. A. Mehta · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…