DeCLIP: Decoding CLIP representations for deepfake localization

Understand

Generative models can create entirely new images, but they can also partially modify real images in ways that are undetectable to the human eye.

• In this paper, we address the challenge of automatically detecting such local manipulations.
• One of the most pressing problems in deepfake detection remains the ability of models to generalize to different classes of generators.
• In the case of fully manipulated images, representations extracted from large self-supervised models (such as CLIP) provide a promising direction towards more robust detectors.