Fetching the paper…

Reading the bibliography…

2018

Uncertainty Estimation via Stochastic Batch Normalization

Atanov, Andrei, Ashukha, Arsenii, Molchanov, Dmitry et al.

Original
PDF
Post on X

Understand

In this work, we investigate Batch Normalization technique and propose its probabilistic interpretation.

We propose a probabilistic model and show that Batch Normalization maximazes the lower bound of its marginalized log-likelihood.
Then, according to the new probabilistic model, we design an algorithm which acts consistently during train and test.
However, inference becomes computationally inefficient.

Uncertainty Estimation via Stochastic Batch Normalization · Around

Built on

A practical bayesian framework for backpropagation networks
David J. C. MacKay · 1992
Earlier work this paper cites.
Bayesian learning via stochastic gradient langevin dynamics
Max Welling and Yee Whye Teh · 2011
Earlier work this paper cites.
Stochastic variational inference
Matthew D. Hoffman, David M. Blei, Chong Wang, and John Paisley · 2013
Earlier work this paper cites.
Dropout: A simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov · 2014
Earlier work this paper cites.

Similar

Dropout as a Bayesian approximation: Representing model uncertainty in deep learning
Original
Yarin Gal and Zoubin Ghahramani · 2015
Cited alongside, same era.
Deep residual learning for image recognition
Original
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Original
Sergey Ioffe and Christian Szegedy · 2015
Cited alongside, same era.
Variational dropout and the local reparameterization trick
Diederik P Kingma, Tim Salimans, and Max Welling · 2015
Cited alongside, same era.

Then

Simple and scalable predictive uncertainty estimation using deep ensembles
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell · 2017
Later among the works it cites.
Multiplicative normalizing flows for variational bayesian neural networks
Christos Louizos and Max Welling · 2017
Later among the works it cites.
Variational dropout sparsifies deep neural networks
Original
Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov · 2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…