Variational Inference based on Robust Divergences

Understand

Robustness to outliers is a central issue in real-world machine learning applications.

• While replacing a model to a heavy-tailed one (e.g., from Gaussian to Student-t) is a standard approach for robustification, it can only be applied to simple models.
• In this paper, based on Zellner's optimization and variational formulation of Bayesian inference, we propose an outlier-robust pseudo-Bayesian variational method by replacing the Kullback-Leibler divergence used for data fitting to a robust divergence such as the beta- and gamma-divergences.
• An advantage of our approach is that superior but complex models such as deep networks can also be handled.