Fetching the paper…
Reading the bibliography…
Large Language Models (LLMs) have the capacity to store and recall facts.
“A synopsis of linguistic theory, 1930-1955”
John Firth · 1957
Earlier work this paper cites.
“Neural networks and physical systems with emergent collective computational abilities.”
John Hopfield · 1982
Earlier work this paper cites.
“The Equivalence of Weak, Strong and Complete Convergence in L1 for Kernel Density Estimates”
Luc Devroye · 1983
Earlier work this paper cites.
“A model of the neural basis of the rat’s sense of direction”
William Skaggs, James Knierim, Hemant Kudrimoti and Bruce McNaughton · 1994
Earlier work this paper cites.
“Theory of orientation tuning in visual cortex.”
Rani Ben-Yishai, R Bar-Or and Haim Sompolinsky · 1995
Earlier work this paper cites.
“How the brain keeps the eyes still”
H Seung · 1996
Earlier work this paper cites.
“Elements of information theory”
Thomas Cover · 1999
Earlier work this paper cites.
“Language Models are Few-Shot Learners”, 2020
Tom. Brown et al · 2005
Earlier work this paper cites.
“Decoupled weight decay regularization”
Ilya Loshchilov and Frank Hutter · 2017
Earlier work this paper cites.
“Attention is all you need”
Ashish Vaswani et al · 2017
Earlier work this paper cites.
“Language models are unsupervised multitask learners”
Alec Radford et al · 2019
Earlier work this paper cites.
“Does learning require memorization? a short tale about a long tail”
Vitaly Feldman · 2020
Earlier work this paper cites.
“What neural networks memorize and why: Discovering the long tail via influence estimation”
Vitaly Feldman and Chiyuan Zhang · 2020
Earlier work this paper cites.
“Associative memory in iterated overparameterized sigmoid autoencoders”
Yibo Jiang and Cengiz Pehlevan · 2020
Earlier work this paper cites.
“How context affects language models’ factual predictions”
Fabio Petroni et al · 2020
Earlier work this paper cites.
“Overparameterized neural networks implement associative memory”
Adityanarayanan Radhakrishnan, Mikhail Belkin and Caroline Uhler · 2020
Earlier work this paper cites.
“Hopfield networks is all you need”
Hubert Ramsauer et al · 2020
Earlier work this paper cites.
“Attention approximates sparse distributed memory”
Trenton Bricken and Cengiz Pehlevan · 2021
Earlier work this paper cites.
“Knowledge neurons in pretrained transformers”
Damai Dai et al · 2021
Earlier work this paper cites.
“Editing factual knowledge in language models”
Nicola De, Wilker Aziz and Ivan Titov · 2021
Earlier work this paper cites.
“A mathematical framework for transformer circuits”
Nelson Elhage et al · 2021
Earlier work this paper cites.
“Causal abstractions of neural networks”
Atticus Geiger, Hanson Lu, Thomas Icard and Christopher Potts · 2021
Earlier work this paper cites.
“LoRA: Low-Rank Adaptation of Large Language Models”, 2021
Edward. Hu et al · 2021
Earlier work this paper cites.
Eric Mitchell et al · 2021
Earlier work this paper cites.
Lalchand Pandia and Allyson Ettinger · 2021
Earlier work this paper cites.
“An explanation of in-context learning as implicit bayesian inference”
Sang Xie, Aditi Raghunathan, Percy Liang and Tengyu Ma · 2021
Earlier work this paper cites.
Giovanni Apruzzese et al · 2022
Earlier work this paper cites.
“What is my math transformer doing?–Three results on interpretability and generalization”
Francois Charton · 2022
Earlier work this paper cites.
“Selection-inference: Exploiting large language models for interpretable logical reasoning”
Antonia Creswell, Murray Shanahan and Irina Higgins · 2022
Earlier work this paper cites.
“What can transformers learn in-context? a case study of simple function classes”
Shivam Garg, Dimitris Tsipras, Percy Liang and Gregory Valiant · 2022
Earlier work this paper cites.
“Inducing causal structure for interpretable neural networks”
Atticus Geiger et al · 2022
Earlier work this paper cites.
“Provable memorization capacity of transformers”
Junghwan Kim, Michelle Kim and Barzan Mozafari · 2022
Earlier work this paper cites.
“Towards understanding grokking: An effective theory of representation learning”
Ziming Liu et al · 2022
Earlier work this paper cites.
“Locating and editing factual associations in GPT”
Kevin Meng, David Bau, Alex Andonian and Yonatan Belinkov · 2022
Earlier work this paper cites.
“Universal hopfield networks: A general framework for single-shot associative memory models”
Beren Millidge et al · 2022
Cited alongside, same era.
“Memory-based model editing at scale”
Eric Mitchell et al · 2022
Cited alongside, same era.
“In-context Learning and Induction Heads”, 2022
Catherine Olsson et al · 2022
Cited alongside, same era.
“In-context learning and induction heads”
Catherine Olsson et al · 2022
Cited alongside, same era.
“Ignore previous prompt: Attack techniques for language models”
F\’abio Perez and Ian Ribeiro · 2022
Cited alongside, same era.
“STanhop: Sparse tandem hopfield model for memory-enhanced time series prediction”
Dennis Wu et al · 2023
Later among the works it cites.
“An llm can fool itself: A prompt-based adversarial attack”
Xilie Xu et al · 2023
Later among the works it cites.
“Making retrieval-augmented language models robust to irrelevant context”
Ori Yoran, Tomer Wolfson, Ori Ram and Jonathan Berant · 2023
Later among the works it cites.
Zihan Zhang et al · 2023
Later among the works it cites.
“In-Context Exemplars as Clues to Retrieving from Large Associative Memory”
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
“Why so toxic? measuring and triggering toxic behavior in open-domain chatbots”
Wai Si et al · 2022
Cited alongside, same era.
“Associative memory of structured knowledge”
Julia Steinberg and Haim Sompolinsky · 2022
Cited alongside, same era.
“Interpretability in the wild: a circuit for indirect object identification in gpt-2 small”
Kevin Wang et al · 2022
Cited alongside, same era.
“Unveiling transformers with lego: a synthetic reasoning task”
Yi Zhang et al · 2022
Cited alongside, same era.
Josh Achiam et al · 2023
Cited alongside, same era.
“Physics of Language Models: Part 3.1, Knowledge Storage and Extraction”, 2023
Zeyuan Allen-Zhu and Yuanzhi Li · 2023
Cited alongside, same era.
“Physics of Language Models: Part 3.2, Knowledge Manipulation”, 2023
Zeyuan Allen-Zhu and Yuanzhi Li · 2023
Cited alongside, same era.
Jiachen Zhao · 2023
Later among the works it cites.
“Can We Edit Factual Knowledge by In-Context Learning?”
Ce Zheng et al · 2023
Later among the works it cites.
“Promptbench: Towards evaluating the robustness of large language models on adversarial prompts”
Kaijie Zhu et al · 2023
Later among the works it cites.
“Universal and Transferable Adversarial Attacks on Aligned Language Models”, 2023
Andy Zou et al · 2023
Later among the works it cites.
“Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws”, 2024
Zeyuan Allen-Zhu and Yuanzhi Li · 2024
Closest in time.
“Transformers as statisticians: Provable in-context learning with in-context algorithm selection”
Yu Bai et al · 2024
Closest in time.
“Birth of a transformer: A memory viewpoint”
Alberto Bietti et al · 2024
Closest in time.
“Learning Associative Memories with Gradient Descent”
Vivien Cabannes, Berfin Simsek and Alberto Bietti · 2024
Closest in time.
“Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models”
Arijit Chowdhury et al · 2024
Closest in time.
“The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains”
Benjamin Edelman et al · 2024
Closest in time.
“From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers”
M Emrullah et al · 2024
Closest in time.
“Finding alignments between interpretable causal variables and distributed neural representations”
Atticus Geiger et al · 2024
Closest in time.
“Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models”
Peter Hase, Mohit Bansal, Been Kim and Asma Ghandeharioun · 2024
Closest in time.
“Outlier-efficient hopfield layers for large transformer-based models”
Jerry Yao-Chieh Hu et al · 2024
Closest in time.
“Nonparametric modern hopfield models”
Jerry Yao-Chieh Hu et al · 2024
Closest in time.
“On computational limits of modern hopfield models: A fine-grained complexity analysis”
Jerry Yao-Chieh Hu, Thomas Lin, Zhao Song and Han Liu · 2024
Closest in time.
“On sparse modern hopfield model”
Jerry Yao-Chieh Hu et al · 2024
Closest in time.
“On the Origins of Linear Representations in Large Language Models”
Yibo Jiang et al · 2024
Closest in time.
“Mechanics of next token prediction with self-attention”
Yingcong Li et al · 2024
Closest in time.
“Decoupled Alignment for Robust Plug-and-Play Adaptation”
Haozheng Luo et al · 2024
Closest in time.
“Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains”, 2024
Ashok Makkuva et al · 2024
Closest in time.
“How Transformers Learn Causal Structure with Gradient Descent”, 2024
Eshaan Nichani, Alex Damian and Jason. Lee · 2024
Closest in time.
“Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models”
Goutham Rajendran et al · 2024
Closest in time.
“Implicit Regularization of Gradient Flow on One-Layer Softmax Attention”
Heejune Sheen, Siyu Chen, Tianhao Wang and Harrison Zhou · 2024
Closest in time.
“Gemma: Open models based on gemini research and technology”
Gemma Team et al · 2024
Closest in time.
“InstructEdit: Instruction-based Knowledge Editing for Large Language Models”
Bozhong Tian et al · 2024
Closest in time.
“Uniform memory retrieval with larger capacity for modern hopfield models”
Dennis Wu, Jerry Yao-Chieh Hu, Teng-Yun Hsiao and Han Liu · 2024
Closest in time.
“Interpretability at scale: Identifying causal mechanisms in alpaca”
Zhengxuan Wu et al · 2024
Closest in time.
“Enhancing Jailbreak Attack Against Large Language Models through Silent Tokens”
Jiahao Yu et al · 2024
Closest in time.
“The clock and the pizza: Two stories in mechanistic explanation of neural networks”
Ziqian Zhong, Ziming Liu, Max Tegmark and Jacob Andreas · 2024
Closest in time.