Neuron-Level Differentiation of Memorization and Generalization in Large Language Models

The perceptron: a probabilistic model for information storage and organization in the brain

Frank Rosenblatt. 1958 · 1958

Earlier work this paper cites.

Brodmann’s’ localisation in the cerebral cortex’

Laurence J Garey. 1999 · 1999

Earlier work this paper cites.

Transformer feed-forward layers are key-value memories

Original

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2020 · 2012

Earlier work this paper cites.

Towards ai-complete question answering: A set of prerequisite toy tasks

Original

Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart Van Merriënboer, Armand Joulin, and Tomas Mikolov. 2015 · 2015

Earlier work this paper cites.

Language models are unsupervised multitask learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019

Earlier work this paper cites.

Memorization vs. generalization: Quantifying data leakage in nlp performance evaluation

Original

Aparna Elangovan, Jiayuan He, and Karin Verspoor. 2021 · 2021

Earlier work this paper cites.

Deduplicating training data makes language models better

Original

Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2021 · 2021

Earlier work this paper cites.

Quantifying memorization across neural language models

Original

Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022 · 2022

Earlier work this paper cites.

Towards reasoning in large language models: A survey

Original

Jie Huang and Kevin Chen-Chuan Chang. 2022 · 2022

Earlier work this paper cites.

Locating and editing factual associations in gpt

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022 · 2022

Earlier work this paper cites.

Memorization without overfitting: Analyzing the training dynamics of large language models

Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. 2022 · 2022

Cited alongside, same era.

Instruction-tuning aligns llms to the human brain

Original

Khai Loong Aw, Syrielle Montariol, Badr AlKhamissi, Martin Schrimpf, and Antoine Bosselut. 2023 · 2023

Cited alongside, same era.

Can llm-generated misinformation be detected?

Original

Canyu Chen and Kai Shu. 2023 · 2023

Cited alongside, same era.

Truth-o-meter: Collaborating with llm in fighting its hallucinations

Boris A Galitsky. 2023 · 2023

Cited alongside, same era.

Teaching arithmetic to small transformers

Original

Nayoung Lee, Kartik Sreenivasan, Jason D Lee, Kangwook Lee, and Dimitris Papailiopoulos. 2023 · 2023

Emergent and predictable memorization in large language models

Stella Biderman, Usvsn Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Anthony, Shivanshu Purohit, and Edward Raff. 2024 · 2024

Closest in time.

Unfamiliar finetuning examples control how language models hallucinate

Original

Katie Kang, Eric Wallace, Claire Tomlin, Aviral Kumar, and Sergey Levine. 2024 · 2024

Closest in time.

Learning, forgetting, remembering: Insights from tracking llm memorization during training

Danny Leybzon and Corentin Kervadec. 2024 · 2024

Closest in time.

Inference-time intervention: Eliciting truthful answers from a language model

Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024 · 2024

Closest in time.

Quantifying in-context reasoning effects and memorization effects in llms

Original

Siyu Lou, Yuntian Chen, Xiaodan Liang, Liang Lin, and Quanshi Zhang. 2024 · 2024