Fetching the paper…

Reading the bibliography…

2017

From optimal transport to generative modeling: the VEGAN cookbook

Bousquet, Olivier, Gelly, Sylvain, Tolstikhin, Ilya et al.

Original
PDF
Post on X

Understand

We study unsupervised generative modeling in terms of the optimal transport (OT) problem between true (but unknown) data distribution $P_X$ and the latent variable model distribution $P_G$.

We show that the OT problem can be equivalently written in terms of probabilistic encoders, which are constrained to match the posterior and prior distributions over the latent space.
When relaxed, this constrained optimization problem leads to a penalized optimal transport (POT) objective, which can be efficiently minimized using stochastic gradient descent by sampling from $P_X$ and $P_G$.
We show that POT for the 2-Wasserstein distance coincides with the objective heuristically employed in adversarial auto-encoders (AAE) (Makhzani et al., 2016), which provides the first theoretical justification for AAEs known to the authors.

From optimal transport to generative modeling: the VEGAN cookbook · Around

Built on

On the transfer of masses (in Russian)
L. Kantorovich · 1942
Earlier work this paper cites.
Divergence measures based on the shannon entropy
J. Lin · 1991
Earlier work this paper cites.
Topics in Optimal Transportation
C. Villani · 2003
Earlier work this paper cites.
Convex Analysis and Nonlinear Optimization: Theory and Examples
J. M. Borwein and A. S. Lewis · 2006
Earlier work this paper cites.
Statistical Decision Theory
F. Liese and K.-J. Miescke · 2008
Earlier work this paper cites.

Similar

Introduction to Nonparametric Estimation
A. B. Tsybakov · 2008
Cited alongside, same era.
Representation learning: A review and new perspectives
Y. Bengio, A. Courville, and P. Vincent · 2013
Cited alongside, same era.
Auto-encoding variational Bayes
D. P. Kingma and M. Welling · 2014
Cited alongside, same era.
Generative adversarial nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio · 2014
Cited alongside, same era.
Adversarial autoencoders
A. Makhzani, J. Shlens, N. Jaitly, and I. Goodfellow · 2016
Cited alongside, same era.

Then

Stochastic optimization for large-scale optimal transport
A. Genevay, M. Cuturi, G. Peyré, and F. R. Bach · 2016
Later among the works it cites.
f-GAN: Training generative neural samplers using variational divergence minimization
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka · 2016
Later among the works it cites.
Improved generator objectives for GANs, 2016
B. Poole, A. Alemi, J. Sohl-Dickstein, and A. Angelova · 2016
Later among the works it cites.
Wasserstein GAN, 2017
M. Arjovsky, S. Chintala, and L. Bottou · 2017
Closest in time.
Adversarial variational bayes: Unifying variational autoencoders and generative adversarial networks, 2017
L. Mescheder, S. Nowozin, and A. Geiger · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…