Fetching the paper…

Reading the bibliography…

2018

On the Importance of Strong Baselines in Bayesian Deep Learning

Mukhoti, Jishnu, Stenetorp, Pontus, Gal, Yarin

Original
PDF
Post on X

Understand

Like all sub-fields of machine learning Bayesian Deep Learning is driven by empirical validation of its theoretical proposals.

Given the many aspects of an experiment it is always possible that minor or even major experimental flaws can slip by both authors and reviewers.
One of the most popular experiments used to evaluate approximate inference techniques is the regression experiment on UCI datasets.
However, in this experiment, models which have been trained to convergence have often been compared with baselines trained only for a fixed number of iterations.

On the Importance of Strong Baselines in Bayesian Deep Learning · Around

Built on

Practical variational inference for neural networks
Graves, A. (2011) · 2011
Earlier work this paper cites.
Probabilistic backpropagation for scalable learning of bayesian neural networks
Hernández-Lobato, J. M. and Adams, R. (2015) · 2015
Earlier work this paper cites.
Deep gaussian processes for regression using approximate expectation propagation
Bui, T., Hernández-Lobato, D., Hernandez-Lobato, J., Li, Y., and Turner, R. (2016) · 2016
Earlier work this paper cites.
Dropout as a Bayesian approximation: Representing model uncertainty in deep learning
Gal, Y. and Ghahramani, Z. (2016) · 2016
Earlier work this paper cites.

Similar

Structured and efficient variational deep learning with matrix gaussian posteriors
Louizos, C. and Welling, M. (2016) · 2016
Cited alongside, same era.
Bayesian optimization with robust bayesian neural networks
Springenberg, J. T., Klein, A., Falkner, S., and Hutter, F. (2016) · 2016
Cited alongside, same era.
Model selection in bayesian neural networks via horseshoe priors
Original
Ghosh, S. and Doshi-Velez, F. (2017) · 2017
Cited alongside, same era.

Then

Deep reinforcement learning that matters
Original
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., and Meger, D. (2017) · 2017
Later among the works it cites.
Learning structured weight uncertainty in bayesian neural networks
Sun, S., Chen, C., and Carin, L. (2017) · 2017
Later among the works it cites.
On the state of the art of evaluation in neural language models
Melis, G., Dyer, C., and Blunsom, P. (2018) · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…