Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisation

Understand

PAC-Bayes learning is an established framework to both assess the generalisation ability of learning algorithms, and design new learning algorithm by exploiting generalisation bounds as training objectives.

• Most of the exisiting bounds involve a \emph{Kullback-Leibler} (KL) divergence, which fails to capture the geometric properties of the loss function which are often useful in optimisation.
• We address this by extending the emerging \emph{Wasserstein PAC-Bayes} theory.
• We develop new PAC-Bayes bounds with Wasserstein distances replacing the usual KL, and demonstrate that sound optimisation guarantees translate to good generalisation abilities.