To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models

Trust in automation: Designing for appropriate reliance

John D Lee and Katrina A See. 2004 · 2004

Earlier work this paper cites.

Monitor alarm fatigue: an integrative review

Maria Cvach. 2012 · 2012

Earlier work this paper cites.

Algorithm aversion: people erroneously avoid algorithms after seeing them err

Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015 · 2015

Earlier work this paper cites.

The Decision Making Tendency Inventory: A new measure to assess maximizing, satisficing, and minimizing

Raffaella Misuraca, Palmira Faraci, Amelia Gangemi, Floriana A Carmeci, and Silvana Miceli. 2015 · 2015

Earlier work this paper cites.

Human decisions and machine predictions

Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2018 · 2018

Earlier work this paper cites.

Believe it or not: designing a human-ai partnership for mixed-initiative fact-checking. In Proceedings of the 31st annual ACM symposium on user interface software and technology . 189–199

An T Nguyen, Aditya Kharosekar, Saumyaa Krishnan, Siddhesh Krishnan, Elizabeth Tate, Byron C Wallace, and Matthew Lease. 2018 · 2018

Earlier work this paper cites.

Explanation in artificial intelligence: Insights from the social sciences

Tim Miller. 2019 · 2019

Earlier work this paper cites.

A systematic review of algorithm aversion in augmented decision making

Jason W Burton, Mari-Klara Stein, and Tina Blegind Jensen. 2020 · 2020

Earlier work this paper cites.

Measuring massive multitask language understanding

Original

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020 · 2020

Earlier work this paper cites.

Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency . 295–305

Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020 · 2020

Earlier work this paper cites.

Does the whole exceed its parts? The effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI conference on human factors in computing systems . 1–16

Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021 · 2021

Earlier work this paper cites.

To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making

Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021 · 2021

Earlier work this paper cites.

Towards a science of human-ai decision making: a survey of empirical studies

Original

Vivian Lai, Chacha Chen, Q Vera Liao, Alison Smith-Renner, and Chenhao Tan. 2021 · 2021

Earlier work this paper cites.

Human-centered explainable ai (xai): From algorithms to user experiences

Original

Q Vera Liao and Kush R Varshney. 2021 · 2021

Earlier work this paper cites.

Human reliance on machine learning models when performance feedback is limited: Heuristics and risks. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1–16

Zhuoran Lu and Ming Yin. 2021 · 2021

Earlier work this paper cites.

Anchoring bias affects mental model formation and user reliance in explainable AI systems. In Proceedings of the 26th International Conference on Intelligent User Interfaces . 340–350

Mahsan Nourani, Chiradeep Roy, Jeremy E Block, Donald R Honeycutt, Tahrima Rahman, Eric Ragan, and Vibhav Gogate. 2021 · 2021

Earlier work this paper cites.

Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI conference on human factors in computing systems . 1–52

Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021 · 2021

Earlier work this paper cites.

Are explanations helpful? A comparative study of the effects of explanations in ai-assisted decision-making. In Proceedings of the 26th International Conference on Intelligent User Interfaces . 318–328

Xinru Wang and Ming Yin. 2021 · 2021

Earlier work this paper cites.

How cognitive biases affect XAI-assisted decision-making: A systematic review. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society . 78–91

Astrid Bertrand, Rafik Belloum, James R Eagan, and Winston Maxwell. 2022 · 2022

Earlier work this paper cites.

Measuring progress on scalable oversight for large language models

Original

Samuel R Bowman, Jeeyoon Hyun, Ethan Perez, Edwin Chen, Craig Pettit, Scott Heiner, Kamilė Lukošiūtė, Amanda Askell, Andy Jones, Anna Chen, et al · 2022

Earlier work this paper cites.

Understanding user reliance on AI in assisted decision-making

Shiye Cao and Chien-Ming Huang. 2022 · 2022

Earlier work this paper cites.

Exploring the effects of machine learning literacy interventions on laypeople’s reliance on machine learning models. In Proceedings of the 27th International Conference on Intelligent User Interfaces . 148–161

Chun-Wei Chiang and Ming Yin. 2022 · 2022

Earlier work this paper cites.

Do people engage cognitively with AI? Impact of AI assistance on incidental learning. In Proceedings of the 27th International Conference on Intelligent User Interfaces . 794–806

Krzysztof Z Gajos and Lena Mamykina. 2022 · 2022

Earlier work this paper cites.

Connecting algorithmic research and usage contexts: a perspective of contextualized evaluation for explainable AI. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing , Vol. 10. 147–159

Q Vera Liao, Yunfeng Zhang, Ronny Luss, Finale Doshi-Velez, and Amit Dhurandhar. 2022 · 2022

Earlier work this paper cites.

Teaching models to express their uncertainty in words

Original

Stephanie Lin, Jacob Hilton, and Owain Evans. 2022 · 2022

Earlier work this paper cites.

Should I follow AI-based advice? Measuring appropriate reliance in human-AI decision-making

Original

Max Schemmer, Patrick Hemmer, Niklas Kühl, Carina Benz, and Gerhard Satzger. 2022 · 2022

Earlier work this paper cites.

Uncalibrated models can improve human-ai collaboration

Kailas Vodrahalli, Tobias Gerstenberg, and James Y Zou. 2022 · 2022

Earlier work this paper cites.

Self-consistency improves chain of thought reasoning in language models

Original

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022 · 2022

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022

Earlier work this paper cites.

Combating misinformation in the age of llms: Opportunities and challenges

Canyu Chen and Kai Shu. 2023 · 2023

Earlier work this paper cites.

Understanding the role of human intuition on reliance in human-AI decision-making with explanations

Valerie Chen, Q Vera Liao, Jennifer Wortman Vaughan, and Gagan Bansal. 2023 · 2023

Earlier work this paper cites.

Investigating the impact of user trust on the adoption and use of ChatGPT: survey analysis

Avishek Choudhury and Hamid Shamszare. 2023 · 2023

Cited alongside, same era.

Something Borrowed: Exploring the Influence of AI-Generated Explanation Text on the Composition of Human Explanations. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems . 1–7

Sharon A Ferguson, Paula Akemi Aoyagui, and Anastasia Kuzminykh. 2023 · 2023

Cited alongside, same era.

In search of verifiability: Explanations rarely enable complementary performance in AI-advised decision making

Raymond Fok and Daniel S Weld. 2023 · 2023

Cited alongside, same era.

Knowing about knowing: An illusion of human competence can hinder appropriate reliance on AI systems. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1–18

Gaole He, Lucie Kuiper, and Ujwal Gadiraju. 2023 · 2023

Cited alongside, same era.

(A) I Am Not a Lawyer, But…: Engaging Legal Experts towards Responsible LLM Policies for Legal Advice. In The 2024 ACM Conference on Fairness, Accountability, and Transparency . 2454–2469

Inyoung Cheong, King Xia, KJ Kevin Feng, Quan Ze Chen, and Amy X Zhang. 2024 · 2024

Closest in time.

Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil’s Advocate. In Proceedings of the 29th International Conference on Intelligent User Interfaces . 103–119

Chun-Wei Chiang, Zhuoran Lu, Zhuoyan Li, and Ming Yin. 2024 · 2024

Closest in time.

Modulating Language Model Experiences through Frictions

Original

Katherine M Collins, Valerie Chen, Ilia Sucholutsky, Hannah Rose Kirk, Malak Sadek, Holli Sargeant, Ameet Talwalkar, Adrian Weller, and Umang Bhatt. 2024 · 2024

Closest in time.

Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributions

Original

Hyo Jin Do, Rachel Ostrand, Justin D Weisz, Casey Dugan, Prasanna Sattigeri, Dennis Wei, Keerthiram Murugesan, and Werner Geyer. 2024 · 2024

Closest in time.