2023

Co-Writing with Opinionated Language Models Affects Users' Views

Jakesch, Maurice, Bhat, Advait, Buschek, Daniel et al.

Understand

If large language models like GPT-3 preferably produce a particular point of view, they may influence people's opinions on an unknown scale.

This study investigates whether a language-model-powered writing assistant that generates some opinions more often than others impacts what users write - and what they think.
In an online experiment, we asked participants (N=1,506) to write a post discussing whether social media is good for society.
Treatment group participants used a language-model-powered writing assistant configured to argue that social media is good or bad for society.

Built on

Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901
Earlier work this paper cites.
Effects of group pressure upon the modification and distortion of judgments
Solomon E Asch. 1951 · 1951
Earlier work this paper cites.
Behavioral study of obedience
Stanley Milgram. 1963 · 1963
Earlier work this paper cites.
The people’s choice
Paul F Lazarsfeld, Bernard Berelson, and Hazel Gaudet. 1968 · 1968
Earlier work this paper cites.
Self-perception theory
Daryl J Bem. 1972 · 1972
Earlier work this paper cites.
The elaboration likelihood model of persuasion
Richard E Petty and John T Cacioppo. 1986 · 1986
Earlier work this paper cites.
Treating ordinal scales as interval scales: an attempt to resolve the controversy
Thomas R Knapp. 1990 · 1990
Earlier work this paper cites.
Computers are social actors. In Proceedings of the SIGCHI conference on Human factors in computing systems . 72–78
Clifford Nass, Jonathan Steuer, and Ellen R Tauber. 1994 · 1994
Earlier work this paper cites.
Humans and automation: Use, misuse, disuse, abuse
Raja Parasuraman and Victor Riley. 1997 · 1997
Earlier work this paper cites.
Persuasive technology: using computers to change what we think and do
Brian J Fogg. 2002 · 2002
Earlier work this paper cites.
Is seeing believing? How recommender system interfaces affect users’ opinions. In Proceedings of the SIGCHI conference on Human factors in computing systems . 585–592
Dan Cosley, Shyong K Lam, Istvan Albert, Joseph A Konstan, and John Riedl. 2003 · 2003
Earlier work this paper cites.
Peer-facilitated eating disorder prevention: A randomized effectiveness trial of cognitive dissonance and media advocacy
Carolyn Black Becker, Lisa M Smith, and Anna C Ciao. 2006 · 2006
Earlier work this paper cites.
The spread of obesity in a large social network over 32 years
Nicholas A Christakis and James H Fowler. 2007 · 2007
Earlier work this paper cites.
Social influence
Lisa Rashotte. 2007 · 2007
Earlier work this paper cites.
The collective dynamics of smoking in a large social network
Nicholas A Christakis and James H Fowler. 2008 · 2008
Earlier work this paper cites.
Richard H. Thaler, Cass R. Sunstein, Nudge: Improving decisions about health, wealth, and happiness
Thomas C Leonard. 2008 · 2008
Earlier work this paper cites.
Social Psychology
D.G. Myers. 2008 · 2008
Earlier work this paper cites.
Inducing attitude change toward online gaming among adolescent players based on dissonance theory: The role of threats and justification of effort
Chin-Sheng Wan and Wen-Bin Chiou. 2010 · 2009
Earlier work this paper cites.
Complacency and bias in human use of automation: An attentional integration
Raja Parasuraman and Dietrich H Manzey. 2010 · 2010
Earlier work this paper cites.
Everyone’s an influencer: quantifying influence on twitter. In Proceedings of the fourth ACM international conference on Web search and data mining . 65–74
Eytan Bakshy, Jake M Hofman, Winter A Mason, and Duncan J Watts. 2011 · 2011
Earlier work this paper cites.
Thinking, fast and slow
Daniel Kahneman. 2011 · 2011
Earlier work this paper cites.
Role of the new media in the Arab Spring
Habibul Haque Khondker. 2011 · 2011
Earlier work this paper cites.
I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience
Alice E Marwick and Danah Boyd. 2011 · 2011
Earlier work this paper cites.
Persuasion in society
Herbert W Simons. 2011 · 2011
Earlier work this paper cites.
Influencing individually: fusing personalization and persuasion
Shlomo Berkovsky, Jill Freyne, and Harri Oinas-Kukkonen. 2012 · 2012
Earlier work this paper cites.
Multidimensional Pareto Optimization of Touchscreen Keyboards for Speed, Familiarity and Improved Spell Checking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12) . Association for Computing Machinery, New York, NY, USA, 2669–2678
Mark Dunlop and John Levine. 2012 · 2012
Earlier work this paper cites.
The structure of online diffusion networks. In Proceedings of the 13th ACM conference on electronic commerce . 623–638
Sharad Goel, Duncan J Watts, and Daniel G Goldstein. 2012 · 2012
Earlier work this paper cites.
Using Controversies for Knowledge Construction: Thinking and Writing about Alternative Medicine. In ICLS
Joachim Kimmerle, Johannes Moskaliuk, Martina Bientzle, Ansgar Thiel, and Ulrike Cress. 2012 · 2012
Earlier work this paper cites.
Both Complete and Correct? Multi-Objective Optimization of Touchscreen Keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14) . Association for Computing Machinery, New York, NY, USA, 2297–2306
Xiaojun Bi, Tom Ouyang, and Shumin Zhai. 2014 · 2014
Earlier work this paper cites.
Perception vs. reality: measuring machine translation post-editing productivity. In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas . 60–72
Federico Gaspari, Antonio Toral, Sudip Kumar Naskar, Declan Groves, and Andy Way. 2014 · 2014
Earlier work this paper cites.
The relevance of algorithms
Tarleton Gillespie. 2014 · 2014
Earlier work this paper cites.
The inviscid text entry rate and its application as a grand goal for mobile text entry. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services (MobileHCI ’14) . Association for Computing Machinery, Toronto, ON, Canada, 335–338
Per Ola Kristensson and Keith Vertanen. 2014 · 2014
Earlier work this paper cites.
Effects of Language Modeling and Its Personalization on Touchscreen Typing Performance. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15) . Association for Computing Machinery, New York, NY, USA, 649–658
Andrew Fowler, Kurt Partridge, Ciprian Chelba, Xiaojun Bi, Tom Ouyang, and Shumin Zhai. 2015 · 2015
Earlier work this paper cites.
Irrational exuberance
Robert J Shiller. 2015 · 2015
Earlier work this paper cites.
VelociTap: Investigating Fast Mobile Text Entry using Sentence-Based Decoding of Touchscreen Keyboard Input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15) . Association for Computing Machinery, Seoul, Republic of Korea, 659–668
Keith Vertanen, Haythem Memmi, Justin Emge, Shyam Reyal, and Per Ola Kristensson. 2015 · 2015
Earlier work this paper cites.
Complacency and automation bias in the use of imperfect automation
Christopher D Wickens, Benjamin A Clegg, Alex Z Vieane, and Angelia L Sebok. 2015 · 2015
Earlier work this paper cites.

Similar

On Suggesting Phrases vs. Predicting Words for Mobile Text Composition. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology . ACM, Tokyo Japan, 603–608
Kenneth C. Arnold, Krzysztof Z. Gajos, and Adam T. Kalai. 2016 · 2016
Cited alongside, same era.
National Tracking Poll #2110047
Morning Consult. 2016 · 2016
Cited alongside, same era.
Does Prediction Really Help in Marathi Text Input? Empirical Analysis of a Longitudinal Study. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (Florence, Italy) (MobileHCI ’16) . Association for Computing Machinery, New York, NY, USA, 35–46
Girish Dalvi, Shashank Ahire, Nagraj Emmadi, Manjiri Joshi, Anirudha Joshi, Sanjay Ghosh, Prasad Ghone, and Narendra Parmar. 2016 · 2016
Cited alongside, same era.
The rise of social bots
Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2016 · 2016
Cited alongside, same era.
WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16) . Association for Computing Machinery, New York, NY, USA, 3817–3821
Mitchell Gordon, Tom Ouyang, and Shumin Zhai. 2016 · 2016
Cited alongside, same era.
Smart Reply: Automated Response Suggestion for Email. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD ’16) . Association for Computing Machinery, New York, NY, USA, 955–964
Anjuli Kannan, Karol Kurach, Sujith Ravi, Tobias Kaufmann, Andrew Tomkins, Balint Miklos, Greg Corrado, Laszlo Lukacs, Marina Ganea, Peter Young, and Vivek Ramavajjala. 2016 · 2016
Cited alongside, same era.
Advertising in social media: a review of empirical evidence
Johannes Knoll. 2016 · 2016
Cited alongside, same era.
A Cost-Benefit Study of Text Entry Suggestion Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16) . Association for Computing Machinery, New York, NY, USA, 83–88
Philip Quinn and Shumin Zhai. 2016 · 2016
Cited alongside, same era.
Troops, trolls and troublemakers: A global inventory of organized social media manipulation
Samantha Bradshaw and Philip Howard. 2017 · 2017
Cited alongside, same era.
Semantics derived automatically from language corpora contain human-like biases
Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017 · 2017
Cited alongside, same era.
The anatomy of Reddit: An overview of academic research
Alexey N Medvedev, Renaud Lambiotte, and Jean-Charles Delvenne. 2017 · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Cited alongside, same era.

Then

AI-Mediated Communication: Definition, Research Agenda, and Ethical Considerations
Jeffrey T Hancock, Mor Naaman, and Karen Levy. 2020 · 2020
Later among the works it cites.
Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study
Svetlana Koltovskaia. 2020 · 2020
Later among the works it cites.
What If an AI Told You That 2+ 2 Is 5? Conformity to Algorithmic Recommendations.. In ICIS
Yotam Liel and Lior Zalmanson. 2020 · 2020
Later among the works it cites.
Ethics in cybersecurity research and practice
Kevin Macnish and Jeroen van der Ham. 2020 · 2020
Later among the works it cites.
Political effects of the internet and social media
Ekaterina Zhuravskaya, Maria Petrova, and Ruben Enikolopov. 2020 · 2020
Later among the works it cites.
A general language assistant as a laboratory for alignment
Original
Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al · 2021
Later among the works it cites.
Spinning Sequence-to-Sequence Models with Meta-Backdoors
Original
Eugene Bagdasaryan and Vitaly Shmatikov. 2021 · 2021
Later among the works it cites.
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency . 610–623
Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021 · 2021
Later among the works it cites.
How do people interact with biased text prediction models while writing?. In Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing . Association for Computational Linguistics, Online, 116–121
Advait Bhat, Saaket Agashe, and Anirudha Joshi. 2021 · 2021
Later among the works it cites.
Humans rely more on algorithms than social influence as a task becomes more difficult
Eric Bogert, Aaron Schecter, and Richard T Watson. 2021 · 2021
Later among the works it cites.
On the opportunities and risks of foundation models
Original
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al · 2021
Later among the works it cites.
Truth, Lies, and Automation
Ben Buchanan, Andrew Lohn, Micah Musser, and Katerina Sedova. 2021 · 2021
Later among the works it cites.
The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21) . Association for Computing Machinery, New York, NY, USA, Article 732, 13 pages
Daniel Buschek, Martin Zürn, and Malin Eiband. 2021 · 2021
Later among the works it cites.
The echo chamber effect on social media
Matteo Cinelli, Gianmarco De Francisci Morales, Alessandro Galeazzi, Walter Quattrociocchi, and Michele Starnini. 2021 · 2021
Later among the works it cites.
Persuasive Natural Language Generation–A Literature Review
Original
Sebastian Duerr and Peter A Gloor. 2021 · 2021
Later among the works it cites.
A survey on bias in deep NLP
Ismael Garrido-Muñoz, Arturo Montejo-Ráez, Fernando Martínez-Santiago, and L Alfonso Ureña-López. 2021 · 2021
Later among the works it cites.
Truthfulqa: Measuring how models mimic human falsehoods
Original
Stephanie Lin, Jacob Hilton, and Owain Evans. 2021 · 2021
Later among the works it cites.
HONEST: Measuring hurtful sentence completion in language models. In The 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Association for Computational Linguistics
Debora Nozza, Federico Bianchi, and Dirk Hovy. 2021 · 2021
Later among the works it cites.
Scaling language models: Methods, analysis & insights from training gopher
Original
Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al · 2021
Later among the works it cites.
Ethical and social risks of harm from language models
Original
Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al · 2021
Later among the works it cites.
Language models are few-shot multilingual learners
Original
Genta Indra Winata, Andrea Madotto, Zhaojiang Lin, Rosanne Liu, Jason Yosinski, and Pascale Fung. 2021 · 2021
Later among the works it cites.
Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing
Advait Bhat, Saaket Agashe, Niharika Mohile, Parth Oberoi, Ravi Jangir, and Anirudha Joshi. 2022 · 2022
Later among the works it cites.
Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
Original
Hai Dang, Karim Benharrak, Florian Lehmann, and Daniel Buschek. 2022 · 2022
Later among the works it cites.
How Different Groups Prioritize Ethical Values for Responsible AI
Original
Maurice Jakesch, Zana Buçinca, Saleema Amershi, and Alexandra Olteanu. 2022a · 2022
Later among the works it cites.
Human Heuristics for AI-Generated Language Are Flawed
Original
Maurice Jakesch, Jeffrey Hancock, and Mor Naaman. 2022b · 2022
Later among the works it cites.
The Ghost in the Machine has an American accent: value conflict in GPT-3
Original
Rebecca L Johnson, Giada Pistilli, Natalia Menédez-González, Leslye Denisse Dias Duran, Enrico Panai, Julija Kalpokiene, and Donald Jay Bertulfo. 2022 · 2022
Later among the works it cites.
” Because AI is 100% right and safe”: User Attitudes and Sources of AI Authority in India. In CHI Conference on Human Factors in Computing Systems . 1–18
Shivani Kapania, Oliver Siy, Gabe Clapper, Azhagu Meena SP, and Nithya Sambasivan. 2022 · 2022
Later among the works it cites.
Can AI Persuade? Examining A Large Language Model’s Ability to Generate Pro-Vaccination Messages
Elise Karinshak, Sunny Liu, Joon Sung Park, and Jeff Hancock. 2022 · 2022
Later among the works it cites.
All the news that’s fit to fabricate: AI-generated text as a tool of media misinformation
Sarah Kreps, R Miles McCain, and Miles Brundage. 2022 · 2022
Later among the works it cites.
CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 388, 19 pages
Mina Lee, Percy Liang, and Qian Yang. 2022 · 2022
Later among the works it cites.
Suggestion Lists vs. Continuous Generation: Interaction Design for Writing with Generative Models on Mobile Devices Affect Text Length, Wording and Perceived Authorship. In Mensch Und Computer 2022 (Darmstadt, Germany) (MuC ’22) . Association for Computing Machinery, New York, NY, USA
Florian Lehmann, Niklas Markert, Hai Dang, and Daniel Buschek. 2022 · 2022
Later among the works it cites.
Guiding frozen language models with learned soft prompts
Brian Lester and Noah Constant. 2022 · 2022
Later among the works it cites.
Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence
Nikhil Singh, Guillermo Bernal, Daria Savchenko, and Elena L. Glassman. 2022 · 2022
Later among the works it cites.
Taxonomy of risks posed by language models. In 2022 ACM Conference on Fairness, Accountability, and Transparency . 214–229
Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, et al · 2022
Later among the works it cites.
AI as an Active Writer: Interaction Strategies with Generated Text in Human-AI Collaborative Fiction Writing 56-65. In IUI Workshops
Daijin Yang, Yanpeng Zhou, Zhiyuan Zhang, Toby Jia-Jun Li, and LC Ray. 2022 · 2022
Later among the works it cites.
Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces (IUI ’22) . Association for Computing Machinery, New York, NY, USA, 841–852
Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022 · 2022
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…