Adaptively Optimised Adaptive Importance Samplers

Understand

We introduce a new class of adaptive importance samplers leveraging adaptive optimisation tools, which we term AdaOAIS.

• We build on Optimised Adaptive Importance Samplers (OAIS), a class of techniques that adapt proposals to improve the mean-squared error of the importance sampling estimators by parameterising the proposal and optimising the $\chi^2$-divergence between the target and the proposal.
• We show that a naive implementation of OAIS using stochastic gradient descent may lead to unstable estimators despite its convergence guarantees.
• To remedy this shortcoming, we instead propose to use adaptive optimisers (such as AdaGrad and Adam) to improve the stability of the OAIS.