Built on

R. Inglehart, M. Basanez, J. Diez-Medrano, L. Halman, and R. Luijkx, “World values surveys and european values surveys, 1981-1984, 1990-1993, and 1995-1997,” Ann Arbor-Michigan, Institute for Social Research, ICPSR version
2000
Earlier work this paper cites.
D. Kistler, C. Thöni, and C. Welzel, “Survey response and observed behavior: Emancipative and secular values predict cooperation, protection of property, and pro-social behavior,” World Values Research Paper
2015
Earlier work this paper cites.
S. Haslanger, “Distinguished lecture: Social structure, narrative and explanation,” Canadian Journal of Philosophy
2015
Earlier work this paper cites.
S. Trepte, L. Reinecke, N. B. Ellison, O. Quiring, M. Z. Yao, and M. Ziegele, “A cross-cultural perspective on the privacy calculus,” Social Media+ Society
2017
Earlier work this paper cites.
N. Garg, L. Schiebinger, D. Jurafsky, and J. Zou, “Word embeddings quantify 100 years of gender and ethnic stereotypes,” Proceedings of the National Academy of Sciences
2018
Earlier work this paper cites.
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692
Original
2019
Earlier work this paper cites.
S. E. Friedman, S. Schmer-Galunder, A. Chen, R. P. Goldman, and M. Ausman, “Gender gaps correlate with gender bias in social media word embeddings.,” in CogSci
2020
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al
2020
Earlier work this paper cites.
C. Haerpfer, R. Inglehart, A. Moreno, C. Welzel, K. Kizilova, D.-M. J., M. Lagos, P. Norris, E. Ponarin, and B. P. et al. (eds.), “Wvs wave 7 (2017-2020),” 2020
2020
Earlier work this paper cites.
J. Dodge, M. Sap, A. Marasovic, W. Agnew, G. Ilharco, D. Groeneveld, and M. Gardner, “Documenting the english colossal clean crawled corpus,” CoRR
Original
2021
Earlier work this paper cites.

Similar

L. Schmitt, “Mapping global ai governance: a nascent regime in a fragmented landscape,” AI and Ethics
2022
Cited alongside, same era.
Y. Li, “Cross-cultural privacy differences,” in Modern socio-technical perspectives on privacy
2022
Cited alongside, same era.
R. L. Johnson, G. Pistilli, N. Menédez-González, L. D. D. Duran, E. Panai, J. Kalpokiene, and D. J. Bertulfo, “The ghost in the machine has an american accent: value conflict in gpt-3,” arXiv preprint arXiv:2203.07785
Original
2022
Cited alongside, same era.
M. Miotto, N. Rossberg, and B. Kleinberg, “Who is gpt-3? an exploration of personality, values and demographics,” arXiv preprint arXiv:2209.14338
Original
2022
Cited alongside, same era.

Then

J. J. Horton, “Large language models as simulated economic agents: What can we learn from homo silicus?,” tech. rep., National Bureau of Economic Research, 2023
2023
Closest in time.
Accessed: 2023-10-01
“Commoncrawl, distribution of languages..” https://commoncrawl.github.io/cc-crawlstatistics/plots/languages · 2023
Closest in time.
Accessed: 2023-09-29
“Models - openai api.” https://platform.openai.com/docs/models · 2023
Closest in time.
Accessed: 2023-09-29
“Llama 2 - meta ai.” https://ai.meta.com/llama/ · 2023
Closest in time.
Accessed: 2023-09-29
“Claude.” https://claude.ai/login · 2023
Closest in time.
R. I. Masoud, Z. Liu, M. Ferianc, P. Treleaven, and M. Rodrigues, “Cultural alignment in large language models: An explanatory analysis based on hofstede’s cultural dimensions,” arXiv preprint arXiv:2309.12342
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2023

Assessing LLMs for Moral Value Pluralism

Benkler, Noam, Mosaphir, Drisana, Friedman, Scott et al.

Understand

The fields of AI current lacks methods to quantitatively assess and potentially alter the moral values inherent in the output of large language models (LLMs).

However, decades of social science research has developed and refined widely-accepted moral value surveys, such as the World Values Survey (WVS), eliciting value judgments from direct questions in various geographies.
We have turned those questions into value statements and use NLP to compute to how well popular LLMs are aligned with moral values for various demographics and cultures.
While the WVS is accepted as an explicit assessment of values, we lack methods for assessing implicit moral and cultural values in media, e.g., encountered in social media, political rhetoric, narratives, and generated by AI systems such as LLMs that are increasingly present in our daily lives.

Built on

R. Inglehart, M. Basanez, J. Diez-Medrano, L. Halman, and R. Luijkx, “World values surveys and european values surveys, 1981-1984, 1990-1993, and 1995-1997,” Ann Arbor-Michigan, Institute for Social Research, ICPSR version
2000
Earlier work this paper cites.
D. Kistler, C. Thöni, and C. Welzel, “Survey response and observed behavior: Emancipative and secular values predict cooperation, protection of property, and pro-social behavior,” World Values Research Paper
2015
Earlier work this paper cites.
S. Haslanger, “Distinguished lecture: Social structure, narrative and explanation,” Canadian Journal of Philosophy
2015
Earlier work this paper cites.
S. Trepte, L. Reinecke, N. B. Ellison, O. Quiring, M. Z. Yao, and M. Ziegele, “A cross-cultural perspective on the privacy calculus,” Social Media+ Society
2017
Earlier work this paper cites.
N. Garg, L. Schiebinger, D. Jurafsky, and J. Zou, “Word embeddings quantify 100 years of gender and ethnic stereotypes,” Proceedings of the National Academy of Sciences
2018
Earlier work this paper cites.
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692
Original
2019
Earlier work this paper cites.
S. E. Friedman, S. Schmer-Galunder, A. Chen, R. P. Goldman, and M. Ausman, “Gender gaps correlate with gender bias in social media word embeddings.,” in CogSci
2020
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al
2020
Earlier work this paper cites.
C. Haerpfer, R. Inglehart, A. Moreno, C. Welzel, K. Kizilova, D.-M. J., M. Lagos, P. Norris, E. Ponarin, and B. P. et al. (eds.), “Wvs wave 7 (2017-2020),” 2020
2020
Earlier work this paper cites.
J. Dodge, M. Sap, A. Marasovic, W. Agnew, G. Ilharco, D. Groeneveld, and M. Gardner, “Documenting the english colossal clean crawled corpus,” CoRR
Original
2021
Earlier work this paper cites.

Similar

L. Schmitt, “Mapping global ai governance: a nascent regime in a fragmented landscape,” AI and Ethics
2022
Cited alongside, same era.
Y. Li, “Cross-cultural privacy differences,” in Modern socio-technical perspectives on privacy
2022
Cited alongside, same era.
R. L. Johnson, G. Pistilli, N. Menédez-González, L. D. D. Duran, E. Panai, J. Kalpokiene, and D. J. Bertulfo, “The ghost in the machine has an american accent: value conflict in gpt-3,” arXiv preprint arXiv:2203.07785
Original
2022
Cited alongside, same era.
M. Miotto, N. Rossberg, and B. Kleinberg, “Who is gpt-3? an exploration of personality, values and demographics,” arXiv preprint arXiv:2209.14338
Original
2022
Cited alongside, same era.

Then

J. J. Horton, “Large language models as simulated economic agents: What can we learn from homo silicus?,” tech. rep., National Bureau of Economic Research, 2023
2023
Closest in time.
Accessed: 2023-10-01
“Commoncrawl, distribution of languages..” https://commoncrawl.github.io/cc-crawlstatistics/plots/languages · 2023
Closest in time.
Accessed: 2023-09-29
“Models - openai api.” https://platform.openai.com/docs/models · 2023
Closest in time.
Accessed: 2023-09-29
“Llama 2 - meta ai.” https://ai.meta.com/llama/ · 2023
Closest in time.
Accessed: 2023-09-29
“Claude.” https://claude.ai/login · 2023
Closest in time.
R. I. Masoud, Z. Liu, M. Ferianc, P. Treleaven, and M. Rodrigues, “Cultural alignment in large language models: An explanatory analysis based on hofstede’s cultural dimensions,” arXiv preprint arXiv:2309.12342
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

N. Benkler, S. Friedman, S. Schmer-Galunder, D. Mosaphir, V. Sarathy, P. Kantharaju, M. D. McLure, and R. P. Goldman, “Cultural value resonance in folktales: A transformer-based analysis with the world value corpus,” in International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation

2022

Cited alongside, same era.

John Wiley & Sons, 2023

I. Van de Poel and L. Royakkers, Ethics, technology, and engineering: An introduction · 2023

Cited alongside, same era.

Cambridge University Press, 2023

G. S. de Oliveira and E. Baggs, Psychology’s WEIRD Problems · 2023

Cited alongside, same era.

Accessed: 2023-10-02

E. Weil, “Sam altman is the oppenheimer of our age.” https://nymag.com/intelligencer/article/sam-altman-artificial-intelligence-openai-profile.html · 2023

Cited alongside, same era.

M. Atari, M. J. Xue, P. S. Park, D. E. Blasi, and J. Henrich, “Which humans?,” Sep 2023

2023

Cited alongside, same era.

“Wvs database.”

Cited in the paper.

2023

Closest in time.

E. Durmus, K. Nyugen, T. I. Liao, N. Schiefer, A. Askell, A. Bakhtin, C. Chen, Z. Hatfield-Dodds, D. Hernandez, N. Joseph, L. Lovitt, S. McCandlish, O. Sikder, A. Tamkin, J. Thamkul, J. Kaplan, J. Clark, and D. Ganguli, “Towards measuring the representation of subjective global opinions in language models,” arXiv preprint arXiv:2306.16388

Original

2023

Closest in time.

H. Kotek, R. Dockum, and D. Q. Sun, “Gender bias and stereotypes in large language models,” arXiv preprint arXiv:2308.14921

Original

2023

Closest in time.

F. Motoki, V. Neto, and V. Rodrigues, “More human than human: measuring chatgpt political bias.,” Public Choice

2023

Closest in time.