Paper

Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis

Struppek, Lukas, Hintersdorf, Dominik, Friedrich, Felix et al.

Understand

Models for text-to-image synthesis, such as DALL-E~2 and Stable Diffusion, have recently drawn a lot of interest from academia and the general public.

These models are capable of producing high-quality images that depict a variety of concepts and styles when conditioned on textual descriptions.
However, these models adopt cultural characteristics associated with specific Unicode scripts from their vast amount of training data, which may not be immediately apparent.
We show that by simply inserting single non-Latin characters in a textual description, common models reflect cultural stereotypes and biases in their generated images.

Built on

Measuring individual differences in implicit cognition: The implicit association test
Greenwald, A., McGhee, D., & Schwartz, J. (1998) · 1998
Earlier work this paper cites.
The homograph attack
Gabrilovich, E. & Gontmakher, A. (2002) · 2002
Earlier work this paper cites.
Bad characters: Imperceptible NLP attacks
Boucher, N., Shumailov, I., Anderson, R., & Papernot, N. (2022) · 2004
Earlier work this paper cites.
A scale of occupational prestige in canada, based on noc major groups
Goyder, J. & Frank, K. (2007) · 2007
Earlier work this paper cites.
Visualizing data using t-sne
van der Maaten, L. & Hinton, G. E. (2008) · 2008
Earlier work this paper cites.
Are gender-neutral queries really gender-neutral? mitigating gender bias in image search
Wang, J., Liu, Y., & Wang, X. E. (2021) · 2008
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009
Earlier work this paper cites.
Microsoft coco: Common objects in context
Lin, T.-Y., Maire, M., Belongie, S. J., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · 2014
Earlier work this paper cites.
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., & Fergus, R. (2014) · 2014
Earlier work this paper cites.
Explaining and harnessing adversarial examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015) · 2015
Earlier work this paper cites.
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · 2017
Earlier work this paper cites.
Gans trained by a two time-scale update rule converge to a local nash equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · 2017
Earlier work this paper cites.
Membership inference attacks against machine learning models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017
Earlier work this paper cites.
Black-box generation of adversarial text sequences to evade deep learning classifiers
Gao, J., Lanchantin, J., Soffa, M. L., & Qi, Y. (2018) · 2018
Earlier work this paper cites.
Decoupled weight decay regularization
Loshchilov, I. & Hutter, F. (2019) · 2019
Earlier work this paper cites.
Do imagenet classifiers generalize to imagenet?
Recht, B., Roelofs, R., Schmidt, L., & Shankar, V. (2019) · 2019
Earlier work this paper cites.
Denoising diffusion probabilistic models
Ho, J., Jain, A., & Abbeel, P. (2020) · 2020
Earlier work this paper cites.
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2020) · 2020
Earlier work this paper cites.
Ten years of attacks on companies using visual impersonation of domain names
Simpson, G., Moore, T., & Clayton, R. (2020) · 2020
Earlier work this paper cites.
Improved techniques for training score-based generative models
Song, Y. & Ermon, S. (2020) · 2020
Earlier work this paper cites.
Multimodal datasets: misogyny, pornography, and malignant stereotypes
Original
Birhane, A., Prabhu, V. U., & Kahembwe, E. (2021) · 2021
Earlier work this paper cites.
Openclip
Ilharco, G., Wortsman, M., Wightman, R., Gordon, C., Carlini, N., Taori, R., Dave, A., Shankar, V., Namkoong, H., Miller, J., Hajishirzi, H., Farhadi, A., & Schmidt, L. (2021) · 2021
Earlier work this paper cites.
Fairness, welfare, and equity in personalized pricing
Kallus, N. & Zhou, A. (2021) · 2021
Earlier work this paper cites.

Similar

Fairness, equality, and power in algorithmic decision-making
Kasy, M. & Abebe, R. (2021) · 2021
Cited alongside, same era.
Improved denoising diffusion probabilistic models
Nichol, A. & Dhariwal, P. (2021) · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021) · 2021
Cited alongside, same era.
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · 2021
Cited alongside, same era.
LAION-400M: open dataset of clip-filtered 400 million image-text pairs
Original
Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarczyk, R., Mullis, C., Katta, A., Coombes, T., Jitsev, J., & Komatsuzaki, A. (2021) · 2021
Cited alongside, same era.
Tackling algorithmic disability discrimination in the hiring process: An ethical, legal and technical analysis
Buyl, M., Cociancig, C., Frattone, C., & Roekens, N. (2022) · 2022
Cited alongside, same era.
Poisoning and backdooring contrastive learning
Carlini, N. & Terzis, A. (2022) · 2022
Cited alongside, same era.
Cross-lingual and multilingual clip
Carlsson, F., Eisen, P., Rekathati, F., & Sahlgren, M. (2022) · 2022
Cited alongside, same era.
Testing relational understanding in text-guided image generation
Original
Conwell, C. & Ullman, T. D. (2022) · 2022
Cited alongside, same era.
Unicode technical report #36, unicode security considerations
Davis, M. & Suignard, M. (2014) · 2022
Cited alongside, same era.
Make-a-scene: Scene-based text-to-image generation with human priors
Gafni, O., Polyak, A., Ashual, O., Sheynin, S., Parikh, D., & Taigman, Y. (2022) · 2022
Cited alongside, same era.
Does clip know my face?
Original
Hintersdorf, D., Struppek, L., Brack, M., Friedrich, F., Schramowski, P., & Kersting, K. (2022) · 2022
Cited alongside, same era.

Then

Photorealistic text-to-image diffusion models with deep language understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E. L., Ghasemipour, S. K. S., Lopes, R. G., Ayan, B. K., Salimans, T., Ho, J., Fleet, D. J., & Norouzi, M. (2022) · 2022
Later among the works it cites.
LAION-5B: an open large-scale dataset for training next generation image-text models
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kundurthy, S., Crowson, K., Schmidt, L., Kaczmarczyk, R., & Jitsev, J. (2022) · 2022
Later among the works it cites.
The unicode standard 15.0.0
Unicode Consortium (2022) · 2022
Later among the works it cites.
Markedness in visual semantic AI
Wolfe, R. & Caliskan, A. (2022) · 2022
Later among the works it cites.
Scaling autoregressive models for content-rich text-to-image generation
Yu, J., Xu, Y., Koh, J. Y., Luong, T., Baid, G., Wang, Z., Vasudevan, V., Ku, A., Yang, Y., Ayan, B. K., Hutchinson, B., Han, W., Parekh, Z., Li, X., Zhang, H., Baldridge, J., & Wu, Y. (2022) · 2022
Later among the works it cites.
The AI index 2022 annual report
Original
Zhang, D., Maslej, N., Brynjolfsson, E., Etchemendy, J., Lyons, T., Manyika, J., Ngo, H., Niebles, J. C., Sellitto, M., Sakhaee, E., Shoham, Y., Clark, J., & Perrault, C. R. (2022) · 2022
Later among the works it cites.
Apa dictionary of psychology
American Psychological Association (2023) · 2023
Later among the works it cites.
Clip retrieval
Beaumont, R. (2021) · 2023
Later among the works it cites.
Improving image generation with better captions
Betker, J., Goh, G., Jing, L., Brooks, T., Wang, J., Li, L., Ouyang, L., Zhuang, J., Lee, J., Guo, Y., Manassra, W., Dhariwal, P., Chu, C., Jiao, Y., & Ramesh, A. (2023) · 2023
Later among the works it cites.
Easily accessible text-to-image generation amplifies demographic stereotypes at large scale
Bianchi, F., Kalluri, P., Durmus, E., Ladhak, F., Cheng, M., Nozza, D., Hashimoto, T., Jurafsky, D., Zou, J., & Caliskan, A. (2023) · 2023
Later among the works it cites.
Stereotypes
Bordalo, P., Coffman, K., Gennaioli, N., & Shleifer, A. (2023) · 2023
Later among the works it cites.
Llava-interactive: An all-in-one demo for image chat, segmentation, generation and editing
Original
Chen, W., Spiridonova, I., Yang, J., Gao, J., & Li, C. (2023) · 2023
Later among the works it cites.
Fair diffusion: Instructing text-to-image generation models on fairness
Friedrich, F., Schramowski, P., Brack, M., Struppek, L., Hintersdorf, D., Luccioni, S., & Kersting, K. (2023) · 2023
Later among the works it cites.
A multilevel analysis of social capital and self-reported health: evidence from seoul, south korea
Han, S., Kim, H., & Lee, H.-S. (2023) · 2023
Later among the works it cites.
Welcome to the new surreal. how ai-generated video is changing film
Heavenarchive, W. D. (2023) · 2023
Later among the works it cites.
Improving sample quality of diffusion models using self-attention guidance
Hong, S., Lee, G., Jang, W., & Kim, S. (2023) · 2023
Later among the works it cites.
BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models
Li, J., Li, D., Savarese, S., & Hoi, S. C. H. (2023) · 2023
Later among the works it cites.
A.i. may help design your favorite video game character
Liao, S. (2023) · 2023
Later among the works it cites.
Stable bias: Analyzing societal representations in diffusion models
Original
Luccioni, A. S., Akiki, C., Mitchell, M., & Jernite, Y. (2023) · 2023
Later among the works it cites.
Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models
Schramowski, P., Brack, M., Deiseroth, B., & Kersting, K. (2023) · 2023
Later among the works it cites.
Rickrolling the artist: Injecting backdoors into text encoders for text-to-image synthesis
Struppek, L., Hintersdorf, D., & Kersting, K. (2023) · 2023
Later among the works it cites.
Altdiffusion: A multilingual text-to-image diffusion model
Original
Ye, F., Liu, G., Wu, X., & Wu, L. (2023) · 2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…