Generalized Variational Inference: Three arguments for deriving new Posteriors

Understand

We advocate an optimization-centric view on and introduce a novel generalization of Bayesian inference.

• Our inspiration is the representation of Bayes' rule as infinite-dimensional optimization problem (Csiszar, 1975; Donsker and Varadhan; 1975, Zellner; 1988).
• First, we use it to prove an optimality result of standard Variational Inference (VI): Under the proposed view, the standard Evidence Lower Bound (ELBO) maximizing VI posterior is preferable to alternative approximations of the Bayesian posterior.
• Next, we argue for generalizing standard Bayesian inference.