Fetching the paper…

Reading the bibliography…

2024

Click2Mask: Local Editing with Dynamic Mask Generation

Regev, Omer, Avrahami, Omri, Lischinski, Dani

Original
PDF
Post on X

Understand

Recent advancements in generative models have revolutionized image generation and editing, making these tasks accessible to non-experts.

This paper focuses on local image editing, particularly the task of adding new content to a loosely specified area.
Existing methods often require a precise mask or a detailed description of the location, which can be cumbersome and prone to errors.
We propose Click2Mask, a novel approach that simplifies the local editing process by requiring only a single point of reference (in addition to the content description).

Click2Mask: Local Editing with Dynamic Mask Generation · Around

Built on

PyTorch: An Imperative Style, High-Performance Deep Learning Library
Original
Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; Desmaison, A.; Köpf, A.; Yang, E.; DeVito, Z.; Raison, M.; Tejani, A.; Chilamkurthy, S.; Steiner, B.; Fang, L.; Bai, J.; and Chintala, S. 2019 · 1912
Earlier work this paper cites.
Contingency Tables Involving Small Numbers and the χ 2 \chi^{2} Test
Yates, F. 1934 · 1934
Earlier work this paper cites.
Denoising diffusion implicit models
Original
Song, J.; Meng, C.; and Ermon, S. 2020 · 2010
Earlier work this paper cites.
Auto-Encoding Variational Bayes
Original
Kingma, D. P.; and Welling, M. 2013 · 2013
Earlier work this paper cites.
Denoising diffusion probabilistic models
Ho, J.; Jain, A.; and Abbeel, P. 2020 · 2020
Earlier work this paper cites.
Emerging Properties in Self-Supervised Vision Transformers
Caron, M.; Touvron, H.; Misra, I.; Jégou, H.; Mairal, J.; Bojanowski, P.; and Joulin, A. 2021 · 2021
Earlier work this paper cites.
Diffusion models beat GANs on image synthesis
Dhariwal, P.; and Nichol, A. 2021 · 2021
Earlier work this paper cites.
Learning Transferable Visual Models From Natural Language Supervision
Original
Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; and Sutskever, I. 2021 · 2021
Earlier work this paper cites.
Blended Diffusion for Text-Driven Editing of Natural Images
Avrahami, O.; Lischinski, D.; and Fried, O. 2022 · 2022
Earlier work this paper cites.
Text2live: Text-driven layered image and video editing
Bar-Tal, O.; Ofri-Amar, D.; Fridman, R.; Kasten, Y.; and Dekel, T. 2022 · 2022
Earlier work this paper cites.
DiffEdit: Diffusion-based semantic image editing with mask guidance
Original
Couairon, G.; Verbeek, J.; Schwenk, H.; and Cord, M. 2022 · 2022
Earlier work this paper cites.
StyleGAN-NADA: CLIP-guided domain adaptation of image generators
Gal, R.; Patashnik, O.; Maron, H.; Bermano, A. H.; Chechik, G.; and Cohen-Or, D. 2022 · 2022
Earlier work this paper cites.

Similar

Prompt-to-prompt image editing with cross attention control
Hertz, A.; Mokady, R.; Tenenbaum, J.; Aberman, K.; Pritch, Y.; and Cohen-Or, D. 2022 · 2022
Cited alongside, same era.
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
Original
Nichol, A.; Dhariwal, P.; Ramesh, A.; Shyam, P.; Mishkin, P.; McGrew, B.; Sutskever, I.; and Chen, M. 2022 · 2022
Cited alongside, same era.
Hierarchical Text-Conditional Image Generation with CLIP Latents
Original
Ramesh, A.; Dhariwal, P.; Nichol, A.; Chu, C.; and Chen, M. 2022 · 2022
Cited alongside, same era.
High-resolution image synthesis with latent diffusion models
Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022 · 2022
Cited alongside, same era.
Photorealistic text-to-image diffusion models with deep language understanding
Saharia, C.; Chan, W.; Saxena, S.; Li, L.; Whang, J.; Denton, E. L.; Ghasemipour, K.; Gontijo Lopes, R.; Karagol Ayan, B.; Salimans, T.; et al. 2022 · 2022

Then

Imagic: Text-Based Real Image Editing with Diffusion Models
Kawar, B.; Zada, S.; Lang, O.; Tov, O.; Chang, H.; Dekel, T.; Mosseri, I.; and Irani, M. 2023 · 2023
Later among the works it cites.
Segment Anything
Original
Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.-Y.; Dollár, P.; and Girshick, R. 2023 · 2023
Later among the works it cites.
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
Original
Liu, S.; Zeng, Z.; Ren, T.; Li, F.; Zhang, H.; Yang, J.; Li, C.; Yang, J.; Su, H.; Zhu, J.; and Zhang, L. 2023 · 2023
Later among the works it cites.
Emu Edit: Precise Image Editing via Recognition and Generation Tasks
Original
Sheynin, S.; Polyak, A.; Singer, U.; Kirstain, Y.; Zohar, A.; Ashual, O.; Parikh, D.; and Taigman, Y. 2023 · 2023
Later among the works it cites.
DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation

Tumanyan, N.; Geyer, M.; Bagon, S.; and Dekel, T. 2022 · 2022

Cited alongside, same era.

SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model

Xie, S.; Zhang, Z.; Lin, Z.; Hinz, T.; and Zhang, K. 2022 · 2022

Cited alongside, same era.

Break-A-Scene: Extracting Multiple Concepts from a Single Image

Avrahami, O.; Aberman, K.; Fried, O.; Cohen-Or, D.; and Lischinski, D. 2023 · 2023

Cited alongside, same era.

Blended Latent Diffusion

Avrahami, O.; Fried, O.; and Lischinski, D. 2023 · 2023

Cited alongside, same era.

Improving Image Generation with Better Captions

Betker, J.; Goh, G.; Jing, L.; Brooks, T.; Wang, J.; Li, L.; Ouyang, L.; Zhuang, J.; Lee, J.; Guo, Y.; Manassra, W.; Dhariwal, P.; Chu, C.; Jiao, Y.; and Ramesh, A. 2023 · 2023

Cited alongside, same era.

InstructPix2Pix: Learning to Follow Image Editing Instructions

Brooks, T.; Holynski, A.; and Efros, A. A. 2023 · 2023

Cited alongside, same era.

X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling

Pearson, K. 1900

Cited in the paper.

Shi, Y.; Xue, C.; Pan, J.; Zhang, W.; Tan, V. Y.; and Bai, S. 2023 · 2023

Later among the works it cites.

Alpha-CLIP: A CLIP Model Focusing on Wherever You Want

Original

Sun, Z.; Fang, Y.; Wu, T.; Zhang, P.; Zang, Y.; Kong, S.; Xiong, Y.; Lin, D.; and Wang, J. 2023 · 2023

Later among the works it cites.

Edit Everything: A Text-Guided Generative System for Images Editing

Original

Xie, D.; Wang, R.; Ma, J.; Chen, C.; Lu, H.; Yang, D.; Shi, F.; and Lin, X. 2023 · 2023

Later among the works it cites.

MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing

Zhang, K.; Mo, L.; Chen, W.; Sun, H.; and Su, Y. 2023 · 2023

Later among the works it cites.

Guiding Instruction-based Image Editing via Multimodal Large Language Models

Fu, T.-J.; Hu, W.; Du, X.; Wang, W. Y.; Yang, Y.; and Gan, Z. 2024 · 2024

Closest in time.

MagicEraser: Erasing Any Objects via Semantics-Aware Control

Original

Li, F.; Zhang, Z.; Huang, Y.; Liu, J.; Pei, R.; Shao, B.; and Xu, S. 2024 · 2024

Closest in time.

Towards Efficient Diffusion-Based Image Editing with Instant Attention Masks

Zou, S.; Tang, J.; Zhou, Y.; He, J.; Zhao, C.; Zhang, R.; Hu, Z.; and Sun, X. 2024 · 2024

Closest in time.