2023

Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models

Shan, Shawn, Cryan, Jenna, Wenger, Emily et al.

Understand

Recent text-to-image diffusion models such as MidJourney and Stable Diffusion threaten to displace many in the professional artist community.

In particular, models can learn to mimic the artistic style of specific artists after "fine-tuning" on samples of their art.
In this paper, we describe the design, implementation and evaluation of Glaze, a tool that enables artists to apply "style cloaks" to their art before sharing online.
These cloaks apply barely perceptible perturbations to images, and when used as training data, mislead generative models that try to mimic a specific artist.

Built on

Improving one-class SVM for anomaly detection
2003
Earlier work this paper cites.
Efficient near-duplicate detection and sub-image retrieval
2004
Earlier work this paper cites.
Numerical optimization, series in operations research and financial engineering
2006
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
2009
Earlier work this paper cites.
How transferable are features in deep neural networks?
2014
Earlier work this paper cites.
Deep learning face attributes in the wild
2015
Earlier work this paper cites.
Generating images from captions with attention
Original
2015
Earlier work this paper cites.
Unsupervised representation learning with deep convolutional generative adversarial networks
Original
2015
Earlier work this paper cites.
Adversarial manipulation of deep representations
Original
2015
Earlier work this paper cites.
Large-scale classification of fine-art paintings: Learning the right metric on the right feature
Original
2015
Earlier work this paper cites.
Adversarial examples in the physical world
Original
2016
Earlier work this paper cites.
Transferability in machine learning: From phenomena to black-box attacks using adversarial samples
Original
2016
Earlier work this paper cites.
Adversarial images for variational autoencoders
Original
2016
Earlier work this paper cites.
Yfcc100m: The new data in multimedia research
2016
Earlier work this paper cites.
Sampling generative networks
Original
2016
Earlier work this paper cites.
Adversarial examples are not easily detected: Bypassing ten detection methods
2017
Earlier work this paper cites.
Towards evaluating the robustness of neural networks
2017
Earlier work this paper cites.
Detecting adversarial samples from artifacts
Original
2017
Earlier work this paper cites.
Deep feature interpolation for image content changes
2017
Earlier work this paper cites.
Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks
2017
Earlier work this paper cites.
Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising
2017
Earlier work this paper cites.
The perception-distortion tradeoff
2018
Earlier work this paper cites.
Adversarial attacks on variational autoencoders
Original
2018
Earlier work this paper cites.
Adversarial examples for generative models
2018
Earlier work this paper cites.
Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning
2018
Earlier work this paper cites.
When does machine learning fail? generalized transferability for evasion and poisoning attacks
2018
Earlier work this paper cites.
Attngan: Fine-grained text to image generation with attentional generative adversarial networks
2018
Earlier work this paper cites.
The unreasonable effectiveness of deep features as a perceptual metric
2018
Earlier work this paper cites.
Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks
2019
Earlier work this paper cites.
Object-driven text-to-image synthesis via adversarial training
2019
Earlier work this paper cites.
Neural cleanse: Identifying and mitigating backdoor attacks in neural networks
2019
Earlier work this paper cites.
Dm-gan: Dynamic memory generative adversarial networks for text-to-image synthesis
2019
Earlier work this paper cites.
Face-off: Adversarial face obfuscation
2020
Earlier work this paper cites.
Training generative adversarial networks with limited data
2020
Earlier work this paper cites.
Perceptual adversarial robustness: Defense against unseen threat models
Original
2020
Earlier work this paper cites.

Similar

Gotta catch’em all: Using honeypots to catch adversarial attacks on neural networks
2020
Cited alongside, same era.
Fawkes: Protecting privacy against unauthorized deep learning models
2020
Cited alongside, same era.
Smooth adversarial examples
2020
Cited alongside, same era.
Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts
2021
Cited alongside, same era.
Lowkey: Leveraging adversarial attacks to protect social media users from facial recognition
2021
Cited alongside, same era.
Dalle mini, 2021
2021
Cited alongside, same era.
Foggysight: A scheme for facial lookup privacy
2021
Cited alongside, same era.
Digital Illustration Styles
2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
2021
Cited alongside, same era.
Data poisoning won’t save you from facial recognition
Original
2021
Cited alongside, same era.
Zero-shot text-to-image generation
2021
Cited alongside, same era.
Augmented lagrangian adversarial attacks
2021
Cited alongside, same era.

Then

Community Showcase, 2022
2022
Later among the works it cites.
Is Lensa AI Stealing From Human Art? An Expert Explains The Controversy
2022
Later among the works it cites.
NovelAI changelog, 2022
2022
Later among the works it cites.
AI Art Generator Cupixel Rakes in 5M From Craft Store JOANN
2022
Later among the works it cites.
AI creating ‘art’ is an ethical and copyright nightmare
2022
Later among the works it cites.
He Used AI to Publish a Children’s Book in a Weekend. Artists Are Not Happy About It
2022
Later among the works it cites.
Hierarchical text-conditional image generation with clip latents
Original
2022
Later among the works it cites.
High-resolution image synthesis with latent diffusion models
2022
Later among the works it cites.
An A.I.-Generated Picture Won an Art Prize. Artists Aren’t Happy
2022
Later among the works it cites.
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation
Original
2022
Later among the works it cites.
Palette: Image-to-image diffusion models
2022
Later among the works it cites.
AI is coming for commercial art jobs. can it be stopped?
2022
Later among the works it cites.
Raising the cost of malicious AI-powered image editing
Original
2022
Later among the works it cites.
AI-generated game assets, 2022
2022
Later among the works it cites.
Laion-5b: An open large-scale dataset for training next generation image-text models
Original
2022
Later among the works it cites.
Post-breach recovery: Protection against white-box adversarial examples for leaked dnn models
2022
Later among the works it cites.
Stable Diffusion v2.1 and DreamStudio Updates, Dec. 2022
2022
Later among the works it cites.
DF-GAN: A simple and effective baseline for text-to-image synthesis
2022
Later among the works it cites.
Image Apps Like Lensa AI Are Sweeping the Internet, and Stealing From Artists
2022
Later among the works it cites.
ArtStation is hiding images protesting AI art on the platform
2022
Later among the works it cites.
People Have Raised Serious Concerns About The AI Art App That’s All Over Your Instagram Feed
2022
Later among the works it cites.
Scenario lands $6M for its AI platform that generates game art assets
2022
Later among the works it cites.
Stability AI, the startup behind Stable Diffusion, raises 101M
2022
Later among the works it cites.
Why Artists are Fed Up with AI Art
2022
Later among the works it cites.
AI-created images lose U.S. copyrights in test for new technology
2023
Closest in time.
Hundreds of animation guild members join WGA writers on picket line for first time
2023
Closest in time.
Meet the three artists behind a landmark lawsuit against AI art generators
2023
Closest in time.
AI art and the problem of consent
2023
Closest in time.
Does ArtStation become PromptStation?
2023
Closest in time.
AI art & the ethical concerns of artists
2023
Closest in time.
Class Action Filed Against Stability AI, Midjourney, and DeviantArt for DMCA Violations, Right of Publicity Violations, Unlawful Competition, Breach of TOS, 2023
2023
Closest in time.
Imagic: Text-based real image editing with diffusion models
2023
Closest in time.
AI is causing student artists to rethink their creative career plans
2023
Closest in time.
Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery
Original
2023
Closest in time.
SoK: Anti-facial recognition technology
2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…