Stochastic Variational Inference via Upper Bound

Understand

Stochastic variational inference (SVI) plays a key role in Bayesian deep learning.

• Recently various divergences have been proposed to design the surrogate loss for variational inference.
• We present a simple upper bound of the evidence as the surrogate loss.
• This evidence upper bound (EUBO) equals to the log marginal likelihood plus the KL-divergence between the posterior and the proposal.