Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations

Understand

Natural-gradient methods enable fast and simple algorithms for variational inference, but due to computational difficulties, their use is mostly limited to \emph{minimal} exponential-family (EF) approximations.

• In this paper, we extend their application to estimate \emph{structured} approximations such as mixtures of EF distributions.
• Such approximations can fit complex, multimodal posterior distributions and are generally more accurate than unimodal EF approximations.
• By using a \emph{minimal conditional-EF} representation of such approximations, we derive simple natural-gradient updates.