2023

Large Language Models Can Be Easily Distracted by Irrelevant Context

Shi, Freda, Chen, Xinyun, Misra, Kanishka et al.

Understand

Large language models have achieved impressive performance on various natural language processing tasks.

However, so far they have been evaluated primarily on benchmarks where all information in the input context is relevant for solving the task.
In this work, we investigate the distractibility of large language models, i.e., how the model problem-solving accuracy can be influenced by irrelevant context.
In particular, we introduce Grade-School Math with Irrelevant Context (GSM-IC), an arithmetic reasoning dataset with irrelevant information in the problem description.

Built on

Effects of varying irrelevant information on adult age differences in problem solving
Hoyer, W. J., Rebok, G. W., and Sved, S. M · 1979
Earlier work this paper cites.
Working memory and intrusions of irrelevant information in a group of specific poor problem solvers
Pasolunghi, M. C., Cornoldi, C., and De Liberto, S · 1999
Earlier work this paper cites.
The disturbing effect of irrelevant information on arithmetic problem solving in inattentive children
Marzocchi, G. M., Lucangeli, D., De Meo, T., Fini, F., and Cornoldi, C · 2002
Earlier work this paper cites.
Towards ai-complete question answering: A set of prerequisite toy tasks
Original
Weston, J., Bordes, A., Chopra, S., Rush, A. M., Van Merriënboer, B., Joulin, A., and Mikolov, T · 2015
Earlier work this paper cites.
Adversarial examples for evaluating reading comprehension systems
Original
Jia, R. and Liang, P · 2017
Earlier work this paper cites.
Learning what is essential in questions
Khashabi, D., Khot, T., Sabharwal, A., and Roth, D · 2017
Earlier work this paper cites.
Program induction by rationale generation: Learning to solve and explain algebraic word problems
Ling, W., Yogatama, D., Dyer, C., and Blunsom, P · 2017
Earlier work this paper cites.
Learning visually-grounded semantics from contrastive adversarial samples
Shi, H., Mao, J., Xiao, T., Jiang, Y., and Sun, J · 2018
Earlier work this paper cites.
Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs
Dua, D., Wang, Y., Dasigi, P., Stanovsky, G., Singh, S., and Gardner, M · 2019
Earlier work this paper cites.
Learning to attend on essential terms: An enhanced retriever-reader model for open-domain question answering
Ni, J., Zhu, C., Chen, W., and McAuley, J · 2019
Earlier work this paper cites.
CLUTRR: A diagnostic benchmark for inductive reasoning from text
Sinha, K., Sodhani, S., Dong, J., Pineau, J., and Hamilton, W. L · 2019
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Earlier work this paper cites.
Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly
Kassner, N. and Schütze, H · 2020
Earlier work this paper cites.
Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp
Original
Morris, J. X., Lifland, E., Yoo, J. Y., Grigsby, J., Jin, D., and Qi, Y · 2020
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J · 2020
Earlier work this paper cites.
Proofwriter: Generating implications, proofs, and abductive statements over natural language
Original
Tafjord, O., Mishra, B. D., and Clark, P · 2020
Earlier work this paper cites.
Program synthesis with large language models
Original
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., and Sutton, C · 2021
Earlier work this paper cites.

Similar

Look at that! BERT can be easily distracted from paying attention to morphosyntax
Chaves, R. P. and Richter, S. N · 2021
Cited alongside, same era.
Transformers as soft reasoners over language
Clark, P., Tafjord, O., and Richardson, K · 2021
Cited alongside, same era.
Training verifiers to solve math word problems
Original
Cobbe, K., Kosaraju, V., Bavarian, M., Hilton, J., Nakano, R., Hesse, C., and Schulman, J · 2021
Cited alongside, same era.
Adversarial examples for evaluating math word problem solvers
Kumar, V., Maheshwary, R., and Pudi, V · 2021
Cited alongside, same era.
Show your work: Scratchpads for intermediate computation with language models
Original
Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., Sutton, C., and Odena, A · 2021
Cited alongside, same era.
Sorting through the noise: Testing robustness of information processing in pre-trained language models
Pandia, L. and Ettinger, A · 2021
Cited alongside, same era.
Are nlp models really able to solve simple math word problems?
Patel, A., Bhattamishra, S., and Goyal, N · 2021
Cited alongside, same era.
Multitask prompted training enables zero-shot task generalization
Original
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al · 2021
Cited alongside, same era.
Adversarial glue: A multi-task benchmark for robustness evaluation of language models
Original
Wang, B., Xu, C., Wang, S., Gan, Z., Cheng, Y., Gao, J., Awadallah, A. H., and Li, B · 2021
Cited alongside, same era.
Do prompt-based models really understand the meaning of their prompts?
Original
Webson, A. and Pavlick, E · 2021
Cited alongside, same era.
Finetuned language models are zero-shot learners
Original
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V · 2021
Cited alongside, same era.
Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks
Original
Chen, W., Ma, X., Wang, X., and Cohen, W. W · 2022
Cited alongside, same era.

Then

Pal: Program-aided language models
Original
Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., and Neubig, G · 2022
Later among the works it cites.
Folio: Natural language reasoning with first-order logic
Original
Han, S., Schoelkopf, H., Zhao, Y., Qi, Z., Riddell, M., Benson, L., Sun, L., Zubova, E., Qiao, Y., Burtell, M., et al · 2022
Later among the works it cites.
Capturing failures of large language models via human cognitive biases
Original
Jones, E. and Steinhardt, J · 2022
Later among the works it cites.
Decomposed prompting: A modular approach for solving complex tasks
Original
Khot, T., Trivedi, H., Finlayson, M., Fu, Y., Richardson, K., Clark, P., and Sabharwal, A · 2022
Later among the works it cites.
Ground-truth labels matter: A deeper look into input-label demonstrations
Original
Kim, J., Kim, H. J., Cho, H., Jo, H., Lee, S.-W., Lee, S.-g., Yoo, K. M., and Kim, T · 2022
Later among the works it cites.
Large language models are zero-shot reasoners
Original
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y · 2022
Later among the works it cites.
Large language models with controllable working memory
Original
Li, D., Rawat, A. S., Zaheer, M., Wang, X., Lukasik, M., Veit, A., Yu, F., and Kumar, S · 2022
Later among the works it cites.
Holistic evaluation of language models
Original
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al · 2022
Later among the works it cites.
Text and patterns: For effective chain of thought, it takes two to tango
Original
Madaan, A. and Yazdanbakhsh, A · 2022
Later among the works it cites.
Rethinking the role of demonstrations: What makes in-context learning work?
Original
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., and Zettlemoyer, L · 2022
Later among the works it cites.
Training language models to follow instructions with human feedback
Original
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al · 2022
Later among the works it cites.
Measuring and narrowing the compositionality gap in language models
Original
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., and Lewis, M · 2022
Later among the works it cites.
Condaqa: A contrastive reading comprehension dataset for reasoning about negation
Original
Ravichander, A., Gardner, M., and Marasović, A · 2022
Later among the works it cites.
Challenging big-bench tasks and whether chain-of-thought can solve them
Original
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., and Wei, J · 2022
Later among the works it cites.
Chain of thought prompting elicits reasoning in large language models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., and Zhou, D · 2022
Later among the works it cites.
Least-to-most prompting enables complex reasoning in large language models
Original
Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Bousquet, O., Le, Q., and Chi, E · 2022
Later among the works it cites.
COMPS: Conceptual minimal pair sentences for testing robust property knowledge and its inheritance in pre-trained language models
Misra, K., Rayz, J., and Ettinger, A · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…