2022

SinDDM: A Single Image Denoising Diffusion Model

Kulikov, Vladimir, Yadin, Shahar, Kleiner, Matan et al.

Understand

Denoising diffusion models (DDMs) have led to staggering performance leaps in image generation, editing and restoration.

However, existing DDMs use very large datasets for training.
Here, we introduce a framework for training a DDM on a single image.
Our method, which we coin SinDDM, learns the internal statistics of the training image by using a multi-scale diffusion process.

Built on

Making a “completely blind” image quality analyzer
Mittal, A., Soundararajan, R., and Bovik, A. C · 2012
Earlier work this paper cites.
Deep unsupervised learning using nonequilibrium thermodynamics
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S · 2015
Earlier work this paper cites.
Nima: Neural image assessment
Talebi, H. and Milanfar, P · 2018
Earlier work this paper cites.
Singan: Learning a generative model from a single natural image
Shaham, T. R., Dekel, T., and Michaeli, T · 2019
Earlier work this paper cites.
Ingan: Capturing and retargeting the” dna” of a natural image
Shocher, A., Bagon, S., Isola, P., and Irani, M · 2019
Earlier work this paper cites.
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y · 2020
Earlier work this paper cites.
Hierarchical patch VAE-GAN: Generating diverse videos from a single sample
Gur, S., Benaim, S., and Wolf, L · 2020
Earlier work this paper cites.
Denoising diffusion probabilistic models
Ho, J., Jain, A., and Abbeel, P · 2020
Earlier work this paper cites.
Diffusion models beat gans on image synthesis
Dhariwal, P. and Nichol, A · 2021
Earlier work this paper cites.
Catch-a-waveform: Learning to generate audio from a single short example
Greshler, G., Shaham, T., and Michaeli, T · 2021
Earlier work this paper cites.
Improved techniques for training single-image GANs
Hinz, T., Fisher, M., Wang, O., and Wermter, S · 2021
Earlier work this paper cites.

Similar

Musiq: Multi-scale image quality transformer
Ke, J., Wang, Q., Wang, Y., Milanfar, P., and Yang, F · 2021
Cited alongside, same era.
Sdedit: Guided image synthesis and editing with stochastic differential equations
Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.-Y., and Ermon, S · 2021
Cited alongside, same era.
Improved denoising diffusion probabilistic models
Nichol, A. Q. and Dhariwal, P · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al · 2021
Cited alongside, same era.
Denoising diffusion implicit models
Song, J., Meng, C., and Ermon, S · 2021
Cited alongside, same era.
Sinir: Efficient general image manipulation with single image reconstruction
Yoo, J. and Chen, Q · 2021
Cited alongside, same era.
Patchwise generative convnet: Training energy-based models from a single natural image for internal learning
Zheng, Z., Xie, J., and Li, P · 2021
Cited alongside, same era.
Text2LIVE: Text-driven layered image and video editing
Bar-Tal, O., Ofri-Amar, D., Fridman, R., Kasten, Y., and Dekel, T · 2022
Cited alongside, same era.
Any-resolution training for high-resolution image synthesis
Chai, L., Gharbi, M., Shechtman, E., Isola, P., and Zhang, R · 2022
Cited alongside, same era.
Generating natural images with direct patch distributions matching
Elnekave, A. and Weiss, Y · 2022
Cited alongside, same era.
An image is worth one word: Personalizing text-to-image generation using textual inversion
Original
Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A. H., Chechik, G., and Cohen-Or, D · 2022
Cited alongside, same era.
Photorealistic text-to-image diffusion models with deep language understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S., Ayan, B., Mahdavi, S., Lopes, R., Salimans, T., Ho, J., Fleet, D., and Norouzi, M
Cited in the paper.

Then

Drop the GAN: In defense of patches nearest neighbors as single image generative models
Granot, N., Feinstein, B., Shocher, A., Bagon, S., and Irani, M · 2022
Closest in time.
Imagen video: High definition video generation with diffusion models
Original
Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al · 2022
Closest in time.
GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models
Nichol, A. Q., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., Mcgrew, B., Sutskever, I., and Chen, M · 2022
Closest in time.
Hierarchical text-conditional image generation with clip latents
Original
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M · 2022
Closest in time.
High-resolution image synthesis with latent diffusion models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B · 2022
Closest in time.
Palette: Image-to-image diffusion models
Saharia, C., Chan, W., Chang, H., Lee, C., Ho, J., Salimans, T., Fleet, D., and Norouzi, M · 2022
Closest in time.
Stylegan-xl: Scaling stylegan to large diverse datasets
Sauer, A., Schwarz, K., and Geiger, A · 2022
Closest in time.
Sindiffusion: Learning a diffusion model from a single natural image
Original
Wang, W., Bao, J., Zhou, W., Chen, D., Chen, D., Yuan, L., and Li, H · 2022
Closest in time.
Learning to generate 3d shapes from a single example
Wu, R. and Zheng, C · 2022
Closest in time.
Sinfusion: Training diffusion models on a single image or video
Nikankin, Y., Haim, N., and Irani, M · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…