Towards Measuring and Modeling "Culture" in LLMs: A Survey

The concept of culture*

Leslie A. White. 1959 · 1959

Earlier work this paper cites.

CrowS-pairs: A challenge dataset for measuring social biases in masked language models

Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020 · 1967

Earlier work this paper cites.

Outline of a Theory of Practice

P. Bourdieu. 1972 · 1972

Earlier work this paper cites.

Culture and social system revisited

Talcott Parsons. 1972 · 1972

Earlier work this paper cites.

The Interpretation Of Cultures

C. Geertz. 1973 · 1973

Earlier work this paper cites.

Development in Judging Moral Issues

J.R. Rest and L. Kohlberg. 1979 · 1979

Earlier work this paper cites.

Theory of Culture

R. Münch, N.J. Smelser, American Sociological Association. Theory Section, Deutsche Gesellschaft fur Soziologie. Sektion Soziologische Theorien, and Deutsche Gesellschaft für Soziologie. Sektion Soziologische Theorien. 1992 · 1992

Earlier work this paper cites.

Workflow from within and without: Technology and cooperative work on the print industry shopfloor

John Bowers, Graham Button, and Wes Sharrock. 1995 · 1995

Earlier work this paper cites.

At home with the technology: an ethnographic study of a set-top-box trial

Jon O’Brien, Tom Rodden, Mark Rouncefield, and John A. Hughes. 1999 · 1999

Earlier work this paper cites.

On defining the cultural heritage

Janet Blake. 2000 · 2000

Earlier work this paper cites.

The Cultural Psychology of Development: One Mind, Many Mentalities , volume 1

Richard Shweder, Jacqueline Goodnow, Giyoo Hatano, Robert LeVine, Hazel Markus, and Peggy Miller. 2007 · 2007

Earlier work this paper cites.

The number of distinct basic values and their structure assessed by pvq–40

Jan Cieciuch and Shalom Schwartz. 2012 · 2012

Earlier work this paper cites.

A Cultural Approach to Interpersonal Communication: Essential Readings

L. Monaghan, J.E. Goodman, and J. Robinson. 2012 · 2012

Earlier work this paper cites.

Cultures and organizations: Software of the mind intercultural cooperation and its importance for survival

Cristina Mora. 2013 · 2013

Earlier work this paper cites.

Peer-to-peer in the workplace: A view from the road

Syed Ishtiaque Ahmed, Nicola J. Bidwell, Himanshu Zade, Srihari H. Muralidhar, Anupama Dhareshwar, Baneen Karachiwala, Cedrick N. Tandong, and Jacki O’Neill. 2016 · 2016

Earlier work this paper cites.

Semantics derived automatically from language corpora contain human-like biases

Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017 · 2017

Earlier work this paper cites.

How should my chatbot interact? a survey on social characteristics in human–chatbot interaction design

Ana Paula Chaves and Marco Aurélio Gerosa. 2019 · 2019

Earlier work this paper cites.

Cross-cultural transfer learning for text classification

Dor Ringel, Gal Lavee, Ido Guy, and Kira Radinsky. 2019 · 2019

Earlier work this paper cites.

Language (technology) is power: A critical survey of “bias” in NLP

Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020 · 2020

Earlier work this paper cites.

Social chemistry 101: Learning to reason about social and moral norms

Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. 2020 · 2020

Earlier work this paper cites.

Making chat at home in the hospital: Exploring chat use by nurses

Naveena Karusala, Ding Wang, and Jacki O’Neill. 2020 · 2020

Earlier work this paper cites.

RiSAWOZ: A large-scale multi-domain Wizard-of-Oz dataset with rich semantic annotations for task-oriented dialogue modeling

Jun Quan, Shian Zhang, Qian Cao, Zizhong Li, and Deyi Xiong. 2020 · 2020

Earlier work this paper cites.

Cultural influences on word meanings revealed through large-scale semantic alignment

Bill Thompson, Se’an G. Roberts, and Gary Lupyan. 2020 · 2020

Earlier work this paper cites.

On the dangers of stochastic parrots: Can language models be too big?

Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021 · 2021

Earlier work this paper cites.

StereoSet: Measuring stereotypical bias in pretrained language models

Moin Nadeem, Anna Bethke, and Siva Reddy. 2021 · 2021

Earlier work this paper cites.

Re-imagining algorithmic fairness in india and beyond

Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Tulsee Doshi, and Vinodkumar Prabhakaran. 2021 · 2021

Earlier work this paper cites.

Large pre-trained language models contain human-like biases of what is right and wrong to do

Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin A. Rothkopf, and Kristian Kersting. 2021 · 2021

Earlier work this paper cites.

A word on machine ethics: A response to jiang et al. (2021)

Original

Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, and Adina Williams. 2021 · 2021

Earlier work this paper cites.

Restoring and attributing ancient texts using deep neural networks

Yannis Assael, Thea Sommerschield, Brendan Shillingford, Mahyar Bordbar, John Pavlopoulos, Maria Chatzipanagiotou, Ion Androutsopoulos, Jonathan Prag, and Nando Freitas. 2022 · 2022

Earlier work this paper cites.

Sapir’s thought-grooves and whorf’s tensors: Reconciling transformer architectures with cultural anthropology

Michael Castelle. 2022 · 2022

Earlier work this paper cites.

Ai ethics and the future of where large language models are heading

Lance Eliot. 2022 · 2022

Earlier work this paper cites.

Joint evs/wvs 2017-2022 dataset (joint evs/wvs)

EVS/WVS. 2022 · 2022

Earlier work this paper cites.

Improving alignment of dialogue agents via targeted human judgements

Original

Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fritz, Jaume Sanchez Elias, Richard Green, Soňa Mokrá, Nicholas Fernando, Boxi Wu, Rachel Foley, Susannah Young, Iason Gabriel, William Isaac, John Mellor, Demis Hassabis, Koray Kavukcuoglu, Lisa Anne Hendricks, and Geoffrey Irving. 2022 · 2022

Earlier work this paper cites.

Challenges and strategies in cross-cultural NLP

Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. 2022 · 2022

Earlier work this paper cites.

Can machines learn morality? the delphi experiment

Original

Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini, and Yejin Choi. 2022 · 2022

Earlier work this paper cites.

The ghost in the machine has an american accent: value conflict in gpt-3

Original

Rebecca L Johnson, Giada Pistilli, Natalia Menédez-González, Leslye Denisse Dias Duran, Enrico Panai, Julija Kalpokiene, and Donald Jay Bertulfo. 2022 · 2022

Earlier work this paper cites.

ArtELingo: A million emotion annotations of WikiArt with emphasis on diversity over language and culture

Youssef Mohamed, Mohamed Abdelfattah, Shyma Alhuwaider, Feifan Li, Xiangliang Zhang, Kenneth Church, and Mohamed Elhoseiny. 2022 · 2022

Earlier work this paper cites.

Annotators with attitudes: How annotator beliefs and identities bias toxic language detection

Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022 · 2022

Earlier work this paper cites.

Using large language models to simulate multiple humans and replicate human subject studies

Original

Gati Aher, Rosa I. Arriaga, and Adam Tauman Kalai. 2023 · 2023

Earlier work this paper cites.

SODAPOP: Open-ended discovery of social biases in social commonsense reasoning models

Haozhe An, Zongxia Li, Jieyu Zhao, and Rachel Rudinger. 2023 · 2023

Earlier work this paper cites.

Social commonsense for explanation and cultural bias discovery

Lisa Bauer, Hanna Tischer, and Mohit Bansal. 2023 · 2023

Earlier work this paper cites.

GD-COMET: A geo-diverse commonsense inference model

Mehar Bhatia and Vered Shwartz. 2023 · 2023

Cited alongside, same era.

Assessing cross-cultural alignment between ChatGPT and human societies: An empirical study

Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023 · 2023

Cited alongside, same era.

Sociocultural norm similarities and differences via situational alignment and explainable textual entailment

Sky CH-Wang, Arkadiy Saakyan, Oliver Li, Zhou Yu, and Smaranda Muresan. 2023 · 2023

Cited alongside, same era.

Toward cultural bias evaluation datasets: The case of Bengali gender, religious, and national identity

Dipto Das, Shion Guha, and Bryan Semaan. 2023 · 2023

Cited alongside, same era.

Building socio-culturally inclusive stereotype resources with community engagement

Original

Sunipa Dev, Jaya Goyal, Dinesh Tewari, Shachi Dave, and Vinodkumar Prabhakaran. 2023 · 2023

Cited alongside, same era.

Navigating cultural chasms: Exploring and unlocking the cultural pov of text-to-image models

Original

Mor Ventura, Eyal Ben-David, Anna Korhonen, and Roi Reichart. 2023 · 2023

Later among the works it cites.

Are personalized stochastic parrots more dangerous? evaluating persona biases in dialogue systems

Yixin Wan, Jieyu Zhao, Aman Chadha, Nanyun Peng, and Kai-Wei Chang. 2023 · 2023

Later among the works it cites.

Seaeval for multilingual foundation models: From cross-lingual alignment to cultural reasoning

Original

Bin Wang, Zhengyuan Liu, Xin Huang, Fangkai Jiao, Yang Ding, Ai Ti Aw, and Nancy F. Chen. 2023 · 2023

Later among the works it cites.

Copal-id: Indonesian language reasoning with local culture and nuances

Original

Haryo Akbarianto Wibowo, Erland Hilman Fuadi, Made Nindyatama Nityasya, Radityo Eko Prasojo, and Alham Fikri Aji. 2023 · 2023

Later among the works it cites.