2022

On the detection of synthetic images generated by diffusion models

Corvi, Riccardo, Cozzolino, Davide, Zingarini, Giada et al.

Understand

Over the past decade, there has been tremendous progress in creating synthetic media, mainly thanks to the development of powerful methods based on generative adversarial networks (GAN).

Very recently, methods based on diffusion models (DM) have been gaining the spotlight.
In addition to providing an impressive level of photorealism, they enable the creation of text-based visual content, opening up new and exciting opportunities in many different application fields, from arts to video games.
On the other hand, this property is an additional asset in the hands of malicious users, who can generate and distribute fake media perfectly adapted to their attacks, posing new challenges to the media forensic community.

Built on

“Probabilistic outputs for support vector machines and comparison to regularized likelihood methods,”
J. Platt, · 1999
Earlier work this paper cites.
“UCID: An uncompressed color image database,”
G. Schaefer and M. Stich, · 2003
Earlier work this paper cites.
“Digital camera identification from sensor pattern noise,”
J. Lukàš, J. Fridrich, and M. Goljan, · 2006
Earlier work this paper cites.
“ImageNet: A large-scale hierarchical image database,”
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, · 2009
Earlier work this paper cites.
“Microsoft COCO: Common objects in context,”
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C L. Zitnick, · 2014
Earlier work this paper cites.
“Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,”
K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, · 2017
Earlier work this paper cites.
“Progressive Growing of GANs for Improved Quality, Stability, and Variation,”
T. Karras, T. Aila, S. Laine, and J. Lehtinen, · 2018
Earlier work this paper cites.
“Large Scale GAN Training for High Fidelity Natural Image Synthesis,”
A. Brock, J. Donahue, and K. Simonyan, · 2018
Earlier work this paper cites.
“Do GANs Leave Artificial Fingerprints?,”
F. Marra, D. Gragnaniello, L. Verdoliva, and G. Poggi, · 2019
Earlier work this paper cites.
“Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints,”
N. Yu, L. Davis, and M. Fritz, · 2019
Earlier work this paper cites.
“Detecting and Simulating Artifacts in GAN Fake Images,”
X. Zhang, S. Karaman, and S.-F. Chang, · 2019
Earlier work this paper cites.
“Media forensics and deepfakes: an overview,”
L. Verdoliva, · 2020
Earlier work this paper cites.

Similar

“Leveraging Frequency Analysis for Deep Fake Image Recognition,”
J. Frank, T. Eisenhofer, L. Schönherr, A. Fischer, D. Kolossa, and T. Holz, · 2020
Cited alongside, same era.
“CNN-generated images are surprisingly easy to spot… for now,”
S.-Y. Wang, O. Wang, R. Zhang, A. Owens, and A. Efros, · 2020
Cited alongside, same era.
“What makes fake images detectable? Understanding properties that generalize,”
L. Chai, D. Bau, S.-N. Lim, and P. Isola, · 2020
Cited alongside, same era.
“Analyzing and improving the image quality of StyleGAN,”
T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, · 2020
Cited alongside, same era.
“Noiseprint: A CNN-based camera model fingerprint,”
D. Cozzolino and L. Verdoliva, · 2020
Cited alongside, same era.
“GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models,”
Original
A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, · 2021
Cited alongside, same era.
“Alias-free generative adversarial networks,”
T. Karras, M. Aittala, S. Laine, E. Härkönen, J. Hellsten, J. Lehtinen, and T. Aila, · 2021
Cited alongside, same era.
“StyleGAN3 Synthetic Image Detection,”
K. Nagano, · 2021
Cited alongside, same era.
“Are GAN generated images easy to detect? A critical analysis of the state-of-the-art,”
D. Gragnaniello, D. Cozzolino, F. Marra, G. Poggi, and L. Verdoliva, · 2021
Cited alongside, same era.
“Taming transformers for high-resolution image synthesis,”
P. Esser, R. Rombach, and B. Ommer, · 2021
Cited alongside, same era.
“DALL-E Mini,” 7 2021
B. Dayma, S. Patil, P. Cuenca, K. Saifullah, T. Abraham, P. Lê Khàc, L. Melas, and R. Ghosh, · 2021
Cited alongside, same era.

Then

“Diffusion models beat GANs on image synthesis,”
P. Dhariwal and A. Nichol, · 2021
Later among the works it cites.
“Lighting (in)consistency of paint by text,”
Original
H. Farid, · 2022
Closest in time.
“Perspective (in)consistency of paint by text,”
Original
H. Farid, · 2022
Closest in time.
“Hierarchical text-conditional image generation with clip latents,”
Original
A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, · 2022
Closest in time.
“Stable diffusion,”
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, · 2022
Closest in time.
“IEEE Video and Image Processing Cup,”
R Corvi, D. Cozzolino, K. Nagano, and L. Verdoliva, · 2022
Closest in time.
“DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Diffusion Models,”
Original
Z. Sha, Z. Li, N. Yu, and Y. Zhang, · 2022
Closest in time.
“Detecting GAN-generated Images by Orthogonal Training of Multiple CNNs,”
S. Mandelli, N. Bonettini, P. Bestagini, and S. Tubaro, · 2022
Closest in time.
“Fusing Global and Local Features for Generalized AI-Synthesized Image Detection,”
Y. Ju, S. Jia, L. Ke, H. Xue, K. Nagano, and S. Lyu, · 2022
Closest in time.
“High-resolution image synthesis with latent diffusion models,”
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, · 2022
Closest in time.
“Efficient geometry-aware 3d generative adversarial networks,”
E. R. Chan, C. Z Lin, M. A Chan, K. Nagano, B. Pan, S. De Mello, O. Gallo, L. J. Guibas, J. Tremblay, S. Khamis, T. Karras, and G. Wetzstein, · 2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…