Fetching the paper…
Reading the bibliography…
Large language models (LLMs) often exhibit systematic preferences for certain answer choices when responding to multiple-choice questions-a behavior known as selection bias.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901
Earlier work this paper cites.
Response strategies for coping with the cognitive demands of attitude measures in surveys
Jon A. Krosnick. 1991 · 1991
Earlier work this paper cites.
Questions and answers in attitude surveys: experiments on question forms, wording, and context
Howard Schuman and Stanley Presser. 1996 · 1996
Earlier work this paper cites.
The impact of candidate name order on election outcomes
Joanne M. Miller and Jon A. Krosnick. 1998 · 1998
Earlier work this paper cites.
Response order effects
Paul J. Lavrakas. 2008 · 2008
Earlier work this paper cites.
Data-free parameter pruning for deep neural networks
Suraj Srinivas and R Venkatesh Babu. 2015 · 2015
Earlier work this paper cites.
Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding
Song Han, Huizi Mao, and William J Dally. 2016 · 2016
Earlier work this paper cites.
Large language models sensitivity to the order of options in multiple-choice questions
Pouya Pezeshkpour and Estevam Hruschka. 2024 · 2017
Earlier work this paper cites.
Conceptnet 5.5: An open multilingual graph of general knowledge
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017 · 2017
Earlier work this paper cites.
Think you have solved question answering? try arc, the ai2 reasoning challenge
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018 · 2018
Earlier work this paper cites.
To prune, or not to prune: exploring the efficacy of pruning for model compression
Michael Zhu and Suyog Gupta. 2017 · 2018
Earlier work this paper cites.
Importance estimation for neural network pruning
Pavlo Molchanov, Arun Mallya, Stephen Tyree, Iuri Frosio, and Jan Kautz. 2019 · 2019
Earlier work this paper cites.
CommonsenseQA: A question answering challenge targeting commonsense knowledge
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019 · 2019
Earlier work this paper cites.
Hellaswag: Can a machine really finish your sentence?
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019 · 2019
Earlier work this paper cites.
Robustbench: a standardized adversarial robustness benchmark
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. 2021 · 2021
Earlier work this paper cites.
Measuring massive multitask language understanding
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 · 2021
Cited alongside, same era.
Pruning convolutional neural networks for resource efficient inference
Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. 2022 · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 · 2022
Cited alongside, same era.
Introducing claude
Anthropic. 2023 · 2023
Cited alongside, same era.
Dola: Decoding by contrasting layers improves factuality in large language models
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R Glass, and Pengcheng He. 2023 · 2023
Cited alongside, same era.
Position: Insights from survey methodology can improve training data
Stephanie Eckman, Barbara Plank, and Frauke Kreuter. 2024 · 2024
Closest in time.
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, et al. 2024 · 2024
Closest in time.
Changing answer order can decrease mmlu accuracy
Vipul Gupta, David Pantoja, Candace Ross, Adina Williams, and Megan Ung. 2024 · 2024
Closest in time.
AnnoLLM: Making large language models to be better crowdsourced annotators
Xingwei He, Zhenghao Lin, Yeyun Gong, A-Long Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, and Weizhu Chen. 2024 · 2024
Closest in time.
Anchored answers: Unravelling positional bias in gpt-2’s multiple-choice questions
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 · 2023
Cited alongside, same era.
Coannotating: Uncertainty-guided work allocation between human and large language models for data annotation
Minzhi Li, Taiwei Shi, Caleb Ziems, Min-Yen Kan, Nancy Chen, Zhengyuan Liu, and Diyi Yang. 2023a · 2023
Cited alongside, same era.
Llm-pruner: On the structural pruning of large language models
Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023 · 2023
Cited alongside, same era.
Crosslingual generalization through multitask finetuning
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. 2023 · 2023
Cited alongside, same era.
Leveraging large language models for multiple choice question answering
Joshua Robinson, Christopher Michael Rytting, and David Wingate. 2023 · 2023
Cited alongside, same era.
Do llms exhibit human-like response biases? a case study in survey design
Lindia Tjuatja, Valerie Chen, Sherry Tongshuang Wu, Ameet Talwalkar, and Graham Neubig. 2023 · 2023
Cited alongside, same era.
Large language models are not fair evaluators
Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023 · 2023
Cited alongside, same era.
Ruizhe Li and Yanjun Gao. 2024 · 2024
Closest in time.
Model sparsity can simplify machine unlearning
Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, PRANAY SHARMA, Sijia Liu, et al. 2024 · 2024
Closest in time.
Teacher-student training for debiasing: General permutation debiasing for large language models
Adian Liusie, Yassir Fathullah, and Mark JF Gales. 2024 · 2024
Closest in time.
Introducing meta llama 3: The most capable openly available llm to date
Meta. 2024 · 2024
Closest in time.
Dissecting language models: Machine unlearning via selective pruning
Nicholas Pochinkov and Nandi Schoots. 2024 · 2024
Closest in time.
Beyond performance: Quantifying and mitigating label bias in llms
Yuval Reif and Roy Schwartz. 2024 · 2024
Closest in time.
Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, and Barbara Plank. 2024 · 2024
Closest in time.
Unveiling selection biases: Exploring order and token sensitivity in large language models
Sheng-Lun Wei, Cheng-Kuang Wu, Hen-Hsen Huang, and Hsin-Hsi Chen. 2024 · 2024
Closest in time.
Strengthened symbol binding makes large language models reliable multiple-choice selectors
Mengge Xue, Zhenyu Hu, Meng Zhao, Liqun Liu, Kuo Liao, Shuang Li, Honglin Han, and Chengguo Yin. 2024 · 2024
Closest in time.
Large language models are not robust multiple choice selectors
Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2024 · 2024
Closest in time.