Fast Black-box Variational Inference through Stochastic Trust-Region Optimization

Understand

We introduce TrustVI, a fast second-order algorithm for black-box variational inference based on trust-region optimization and the reparameterization trick.

• At each iteration, TrustVI proposes and assesses a step based on minibatches of draws from the variational distribution.
• The algorithm provably converges to a stationary point.
• We implemented TrustVI in the Stan framework and compared it to two alternatives: Automatic Differentiation Variational Inference (ADVI) and Hessian-free Stochastic Gradient Variational Inference (HFSGVI).