Representation Bias in Political Sample Simulations with Large Language Models

P. Gronke, E. Galanes-Rosenbaum, P. A. Miller, and D. Toffey, “Convenience voting,” Annu. Rev. Polit. Sci. , vol. 11, no. 1, pp. 437–455, 2008

2008

Earlier work this paper cites.

C. Stewart III, “Voting technologies,” Annual Review of Political Science , vol. 14, no. 1, pp. 353–378, 2011

2011

Earlier work this paper cites.

A. J. Berinsky, “Measuring public opinion with surveys,” Annual review of political science , vol. 20, no. 1, pp. 309–329, 2017

2017

Earlier work this paper cites.

J. Pan and Y. Xu, “China’s ideological spectrum,” The Journal of Politics , vol. 80, no. 1, pp. 254–273, 2018

2018

Earlier work this paper cites.

S. Bird, M. Dudík, R. Edgar, B. Horn, R. Lutz, V. Milan, M. Sameki, H. Wallach, and K. Walker, “Fairlearn: A toolkit for assessing and improving fairness in ai,” Microsoft, Tech. Rep. MSR-TR-2020-32 , 2020

2020

Earlier work this paper cites.

E. Ntoutsi, P. Fafalios, U. Gadiraju, V. Iosifidis, W. Nejdl, M.-E. Vidal, S. Ruggieri, F. Turini, S. Papadopoulos, E. Krasanakis et al. , “Bias in data-driven artificial intelligence systems—an introductory survey,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , vol. 10, no. 3, p. e1356, 2020

2020

Earlier work this paper cites.

M. Madaio, L. Egede, H. Subramonyam, J. Wortman Vaughan, and H. Wallach, “Assessing the fairness of ai systems: Ai practitioners’ processes, challenges, and needs for support,” Proceedings of the ACM on Human-Computer Interaction , vol. 6, no. CSCW1, pp. 1–26, 2022

2022

Earlier work this paper cites.

L. P. Argyle, E. C. Busby, N. Fulda, J. R. Gubler, C. Rytting, and D. Wingate, “Out of one, many: Using language models to simulate human samples,” Political Analysis , vol. 31, no. 3, pp. 337–351, 2023

2023

Earlier work this paper cites.

J. J. Horton, “Large language models as simulated economic agents: What can we learn from homo silicus?” National Bureau of Economic Research, Tech. Rep., 2023

2023

Earlier work this paper cites.

C. Ziems, W. Held, O. Shaikh, J. Chen, Z. Zhang, and D. Yang, “Can large language models transform computational social science?” Computational Linguistics , pp. 1–53, 2023

2023

Earlier work this paper cites.

G. V. Aher, R. I. Arriaga, and A. T. Kalai, “Using large language models to simulate multiple humans and replicate human subject studies,” in International Conference on Machine Learning . PMLR, 2023, pp. 337–371

2023

Earlier work this paper cites.

D. Dillion, N. Tandon, Y. Gu, and K. Gray, “Can ai language models replace human participants?” Trends in Cognitive Sciences , 2023

2023

Cited alongside, same era.

J. Hartmann, J. Schwenzow, and M. Witte, “The political ideology of conversational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation,” arXiv preprint arXiv:2301.01768 , 2023

Original

2023

Cited alongside, same era.

H. Kotek, R. Dockum, and D. Sun, “Gender bias and stereotypes in large language models,” in Proceedings of The ACM Collective Intelligence Conference , 2023, pp. 12–24

2023

Cited alongside, same era.

C. Gao, X. Lan, Z. Lu, J. Mao, J. Piao, H. Wang, D. Jin, and Y. Li, “ s 3 s^{3} : Social-network simulation system with large language model-empowered agents,” arXiv preprint arXiv:2307.14984 , 2023

Original

2023

Cited alongside, same era.

Y. Bang, S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung et al. , “A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity,” arXiv preprint arXiv:2302.04023 , 2023

Original

2023

Later among the works it cites.

Y. Zhu, P. Zhang, E.-U. Haq, P. Hui, and G. Tyson, “Can chatgpt reproduce human-generated labels? a study of social computing tasks,” arXiv preprint arXiv:2304.10145 , 2023

Original

2023

Later among the works it cites.

F. Huang, H. Kwak, and J. An, “Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech,” in Companion proceedings of the ACM web conference 2023 , 2023, pp. 294–297

2023

Later among the works it cites.

H. Lyu, S. Jiang, H. Zeng, Y. Xia, Q. Wang, S. Zhang, R. Chen, C. Leung, J. Tang, and J. Luo, “Llm-rec: Personalized recommendation via prompting large language models,” in Findings of the Association for Computational Linguistics: NAACL 2024 , 2024, pp. 583–612