2022

Understanding DDPM Latent Codes Through Optimal Transport

Khrulkov, Valentin, Ryzhakov, Gleb, Chertkov, Andrei et al.

Understand

Diffusion models have recently outperformed alternative approaches to model the distribution of natural images, such as GANs.

Such diffusion models allow for deterministic sampling via the probability flow ODE, giving rise to a latent space and an encoder map.
While having important practical applications, such as estimation of the likelihood, the theoretical properties of this map are not yet fully understood.
In the present work, we partially address this question for the popular case of the VP SDE (DDPM) approach.

Built on

A bound for the error in the normal approximation to the distribution of a sum of dependent random variables
Charles Stein · 1972
Earlier work this paper cites.
The fréchet distance between multivariate normal distributions
DC Dowson and BV666017 Landau · 1982
Earlier work this paper cites.
The distance between two random vectors with given dispersion matrices
Ingram Olkin and Friedrich Pukelsheim · 1982
Earlier work this paper cites.
A class of Wasserstein metrics for probability distributions
Clark Givens and Rae Shortt · 1984
Earlier work this paper cites.
A first course in computational physics
Paul L DeVries and Robert P Wolf · 1994
Earlier work this paper cites.
Double degenerate diffusion equations as steepest descent
Felix Otto · 1996
Earlier work this paper cites.
Fokker-Planck equation
Hannes Risken · 1996
Earlier work this paper cites.
A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem
Jean-David Benamou and Yann Brenier · 2000
Earlier work this paper cites.
Spectral methods in MATLAB , volume 10
Lloyd N Trefethen · 2000
Earlier work this paper cites.
The geometry of dissipative evolution equations: the porous medium equation
Felix Otto · 2001
Earlier work this paper cites.
Optimal transportation, dissipative pde’s and functional inequalities
Cedric Villani · 2003
Earlier work this paper cites.
Optimal transport: old and new , volume 338
Cédric Villani · 2003
Earlier work this paper cites.
Gradient flows: in metric spaces and in the space of probability measures
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré · 2005
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Tt-cross approximation for multidimensional arrays
Ivan Oseledets and Eugene Tyrtyshnikov · 2010
Earlier work this paper cites.
Tensor-train decomposition
Ivan Oseledets · 2011
Earlier work this paper cites.

Similar

Sinkhorn distances: Lightspeed computation of optimal transport
Marco Cuturi · 2013
Cited alongside, same era.
Deep unsupervised learning using nonequilibrium thermodynamics
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli · 2015
Cited alongside, same era.
On the relation between optimal transport and schrödinger bridges: A stochastic control viewpoint
Yongxin Chen, Tryphon T Georgiou, and Michele Pavon · 2016
Cited alongside, same era.
Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions
Andrzej Cichocki, Namgil Lee, Ivan Oseledets, Anh-Huy Phan, Qibin Zhao, and Danilo Mandic · 2016
Cited alongside, same era.
A kernelized Stein discrepancy for goodness-of-fit tests
Qiang Liu, Jason Lee, and Michael Jordan · 2016
Cited alongside, same era.
Near-linear time approximation algorithms for optimal transport via sinkhorn iteration
Jason Altschuler, Jonathan Niles-Weed, and Philippe Rigollet · 2017
Cited alongside, same era.
Tensor networks for dimensionality reduction and large-scale optimization: Part 2 applications and future perspectives
Andrzej Cichocki, Anh Phan, Qibin Zhao, Namgil Lee, Ivan Oseledets, Masashi Sugiyama, and Danilo Mandic · 2017
Cited alongside, same era.
Splitting methods in communication, imaging, science, and engineering
Roland Glowinski, Stanley J Osher, and Wotao Yin · 2017
Cited alongside, same era.
Measuring sample quality with Stein’s method
Jackson Gorham · 2017
Cited alongside, same era.
Gans trained by a two time-scale update rule converge to a local nash equilibrium
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter · 2017
Cited alongside, same era.
Introduction to optimal transport, 2018
Matthew Thorpe · 2018
Cited alongside, same era.
Large scale GAN training for high fidelity natural image synthesis
Andrew Brock, Jeff Donahue, and Karen Simonyan · 2019
Cited alongside, same era.

Then

Stargan v2: Diverse image synthesis for multiple domains
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha · 2020
Later among the works it cites.
Denoising Diffusion Probabilistic Models
Jonathan Ho, Ajay Jain, and Pieter Abbeel · 2020
Later among the works it cites.
Training generative adversarial networks with limited data
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila · 2020
Later among the works it cites.
Solution of the Fokker–Planck Equation by Cross Approximation Method in the Tensor Train Format
Andrei Chertkov and Ivan Oseledets · 2021
Later among the works it cites.
ILVR: Conditioning method for denoising diffusion probabilistic models
Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon · 2021
Later among the works it cites.
Diffusion schrödinger bridge with applications to score-based generative modeling
Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet · 2021
Later among the works it cites.
Diffusion models beat GANs on image synthesis
Prafulla Dhariwal and Alexander Quinn Nichol · 2021
Later among the works it cites.
POT: Python Optimal Transport
Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalie T.H. Gayraud, Hicham Janati, Alain Rakotomamonjy, Ievgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica J. Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer · 2021
Later among the works it cites.
SDEdit: Image synthesis and editing with stochastic differential equations
Original
Chenlin Meng, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon · 2021
Later among the works it cites.
Large-scale wasserstein gradient flows
Petr Mokrov, Alexander Korotin, Lingxiao Li, Aude Genevay, Justin M Solomon, and Evgeny Burnaev · 2021
Later among the works it cites.
Improved denoising diffusion probabilistic models
Alex Nichol and Prafulla Dhariwal · 2021
Later among the works it cites.
Ot-flow: Fast and accurate continuous normalizing flows via optimal transport
Derek Onken, Samy Wu Fung, Xingjian Li, and Lars Ruthotto · 2021
Later among the works it cites.
Image super-resolution via iterative refinement
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi · 2021
Later among the works it cites.
Entropic neural optimal transport via diffusion processes
Original
Nikita Gushchin, Alexander Kolesov, Alexander Korotin, Dmitry Vetrov, and Evgeny Burnaev · 2022
Closest in time.
The flow map of the fokker–planck equation does not provide optimal transport
Hugo Lavenant and Filippo Santambrogio · 2022
Closest in time.
SRDiff: Single Image Super-resolution with Diffusion Probabilistic Models
Haoying Li, Yifan Yang, Meng Chang, Shiqi Chen, Huajun Feng, Zhihai Xu, Qi Li, and Yueting Chen · 2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…