On Implicit Regularization in $\beta$-VAEs

Understand

While the impact of variational inference (VI) on posterior inference in a fixed generative model is well-characterized, its role in regularizing a learned generative model when used in variational autoencoders (VAEs) is poorly understood.

• We study the regularizing effects of variational distributions on learning in generative models from two perspectives.
• First, we analyze the role that the choice of variational family plays in imparting uniqueness to the learned model by restricting the set of optimal generative models.
• Second, we study the regularization effect of the variational family on the local geometry of the decoding model.