2024

Towards Effective User Attribution for Latent Diffusion Models via Watermark-Informed Blending

Pan, Yongyang, Liu, Xiaohong, Luo, Siqi et al.

Understand

Rapid advancements in multimodal large language models have enabled the creation of hyper-realistic images from textual descriptions.

However, these advancements also raise significant concerns about unauthorized use, which hinders their broader distribution.
Traditional watermarking methods often require complex integration or degrade image quality.
To address these challenges, we introduce a novel framework Towards Effective user Attribution for latent diffusion models via Watermark-Informed Blending (TEAWIB).

Built on

Image quality assessment: from error visibility to structural similarity
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli · 2004
Earlier work this paper cites.
Combined dwt-dct digital image watermarking
Ali Al-Haj · 2007
Earlier work this paper cites.
Reversible image watermarking based on integer-to-integer wavelet transform
Sunil Lee, Chang D. Yoo, and Ton Kalker · 2007
Earlier work this paper cites.
An application of multidimensional time-frequency analysis as a base for the unified watermarking approach
Srdjan Stankovic, Irena Orovic, and Nikola Zaric · 2010
Earlier work this paper cites.
Microsoft COCO: Common Objects in Context
Original
Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár · 2014
Earlier work this paper cites.
Decoupled Weight Decay Regularization
Original
Ilya Loshchilov and Frank Hutter · 2017
Earlier work this paper cites.
Variational image compression with a scale hyperprior
Original
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston · 2018
Earlier work this paper cites.
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Original
Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, Hyrum Anderson, Heather Roff, Gregory C. Allen, Jacob Steinhardt, Carrick Flynn, Seán Ó hÉigeartaigh, Simon Beard, Haydn Belfield, Sebastian Farquhar, Clare Lyle, Rebecca Crootof, Owain Evans, Michael Page, Joanna Bryson, Roman Yampolskiy, and Dario Amodei · 2018
Earlier work this paper cites.
Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter · 2018
Earlier work this paper cites.
A Style-Based Generator Architecture for Generative Adversarial Networks
Original
Tero Karras, Samuli Laine, and Timo Aila · 2018
Earlier work this paper cites.
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Original
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang · 2018
Earlier work this paper cites.

Similar

HiDDeN: Hiding Data With Deep Networks
Original
Jiren Zhu, Russell Kaplan, Justin Johnson, and Li Fei-Fei · 2018
Cited alongside, same era.
Analyzing and Improving the Image Quality of StyleGAN
Original
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila · 2019
Cited alongside, same era.
Learned image compression with discretized gaussian mixture likelihoods and attention modules
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto · 2020
Cited alongside, same era.
A Loss Function for Generative Neural Networks Based on Watson’s Perceptual Model
Original
Steffen Czolbe, Oswin Krause, Ingemar Cox, and Christian Igel · 2020
Cited alongside, same era.
Training Generative Adversarial Networks with Limited Data
Original
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila · 2020
Cited alongside, same era.
Distortion agnostic deep watermarking, 2020
Xiyang Luo, Ruohan Zhan, Huiwen Chang, Feng Yang, and Peyman Milanfar · 2020
Cited alongside, same era.
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
Original
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen · 2021
Cited alongside, same era.
High-Resolution Image Synthesis with Latent Diffusion Models
Original
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer · 2021
Cited alongside, same era.
Robust high-capacity watermarking over online social network shared images
Weiwei Sun, Jiantao Zhou, Yuanman Li, Ming Cheung, and James She · 2021
Cited alongside, same era.
Fixed neural network steganography: Train the images, not the network
Varsha Kishore, Xiangyu Chen, Yan Wang, Boyi Li, and Kilian Q Weinberger · 2022
Cited alongside, same era.
Pscc-net: Progressive spatio-channel correlation network for image manipulation detection and localization, 2022
Xiaohong Liu, Yaojie Liu, Jun Chen, and Xiaoming Liu · 2022
Cited alongside, same era.

Then

Hierarchical Text-Conditional Image Generation with CLIP Latents
Original
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen · 2022
Later among the works it cites.
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Original
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi · 2022
Later among the works it cites.
The Stable Signature: Rooting Watermarks in Latent Diffusion Models
Original
Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon · 2023
Later among the works it cites.
Rawiw: Raw image watermarking robust to isp pipeline, 2023
Kang Fu, Xiaohong Liu, Jun Jia, Zicheng Zhang, Yicong Peng, Jia Wang, and Guangtao Zhai · 2023
Later among the works it cites.
Hierarchical fine-grained image forgery detection and localization, 2023
Xiao Guo, Xiaohong Liu, Zhiyuan Ren, Steven Grosz, Iacopo Masi, and Xiaoming Liu · 2023
Later among the works it cites.
Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust
Original
Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein · 2023
Later among the works it cites.
Cheap-fake detection with llm using prompt engineering, 2023
Guangyang Wu, Weijie Wu, Xiaohong Liu, Kele Xu, Tianjiao Wan, and Wenyi Wang · 2023
Later among the works it cites.
WOUAF: Weight Modulation for User Attribution and Fingerprinting in Text-to-Image Diffusion Models
Original
Changhoon Kim, Kyle Min, Maitreya Patel, Sheng Cheng, and Yezhou Yang · 2024
Closest in time.
Diffstega: Towards universal training-free coverless image steganography with diffusion models, 2024
Yiwei Yang, Zheyuan Liu, Jun Jia, Zhongpai Gao, Yunhao Li, Wei Sun, Xiaohong Liu, and Guangtao Zhai · 2024
Closest in time.
Robust Image Watermarking using Stable Diffusion
Original
Lijun Zhang, Xiao Liu, Antoni Viros Martin, Cindy Xiong Bearfield, Yuriy Brun, and Hui Guan · 2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…