2023

Birth of a Transformer: A Memory Viewpoint

Bietti, Alberto, Cabannes, Vivien, Bouchacourt, Diane et al.

Understand

Large language models based on transformers have achieved great empirical successes.

However, as they are deployed more widely, there is a growing need to better understand their internal mechanisms in order to make them more reliable.
These models appear to store vast amounts of knowledge from their training data, and to adapt quickly to new information provided in their context or prompt.
We study how transformers balance these two types of knowledge by considering a synthetic setup where tokens are generated from either global or context-specific bigram distributions.

Built on

Non-holographic associative memory
D. J. Willshaw, O. P. Buneman, and H. C. Longuet-Higgins · 1969
Earlier work this paper cites.
Correlation matrix memories
T. Kohonen · 1972
Earlier work this paper cites.
Neural networks and physical systems with emergent collective computational abilities
J. J. Hopfield · 1982
Earlier work this paper cites.
The capacity of the hopfield associative memory
R. McEliece, E. Posner, E. Rodemich, and S. Venkatesh · 1987
Earlier work this paper cites.
Learning to control fast-weight memories: An alternative to dynamic recurrent networks
J. Schmidhuber · 1992
Earlier work this paper cites.
Efficient estimation of word representations in vector space
T. Mikolov, K. Chen, G. Corrado, and J. Dean · 2013
Earlier work this paper cites.
Neural turing machines
Original
A. Graves, G. Wayne, and I. Danihelka · 2014
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
A. M. Saxe, J. L. McClelland, and S. Ganguli · 2014
Earlier work this paper cites.
Inferring algorithmic patterns with stack-augmented recurrent nets
A. Joulin and T. Mikolov · 2015
Earlier work this paper cites.
End-to-end memory networks
S. Sukhbaatar, A. Szlam, J. Weston, and R. Fergus · 2015
Earlier work this paper cites.
Memory networks
J. Weston, S. Chopra, and A. Bordes · 2015
Earlier work this paper cites.
Layer normalization
Original
J. L. Ba, J. R. Kiros, and G. E. Hinton · 2016
Earlier work this paper cites.
Dense associative memory for pattern recognition
D. Krotov and J. J. Hopfield · 2016
Earlier work this paper cites.
Implicit regularization in matrix factorization
S. Gunasekar, B. E. Woodworth, S. Bhojanapalli, B. Neyshabur, and N. Srebro · 2017
Earlier work this paper cites.
Memory vectors for similarity search in high-dimensional spaces
A. Iscen, T. Furon, V. Gripon, M. Rabbat, and H. Jégou · 2017
Earlier work this paper cites.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin · 2017
Earlier work this paper cites.
Neural tangent kernel: Convergence and generalization in neural networks
A. Jacot, F. Gabriel, and C. Hongler · 2018
Earlier work this paper cites.
High-dimensional probability: An introduction with applications in data science
R. Vershynin · 2018
Earlier work this paper cites.
On lazy training in differentiable programming
L. Chizat, E. Oyallon, and F. Bach · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al · 2019
Earlier work this paper cites.
Root mean square layer normalization
B. Zhang and R. Sennrich · 2019
Earlier work this paper cites.

Similar

Language models are few-shot learners
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al · 2020
Cited alongside, same era.
Infinite attention: Nngp and ntk for deep attention networks
J. Hron, Y. Bahri, J. Sohl-Dickstein, and R. Novak · 2020
Cited alongside, same era.
Associative memory in iterated overparameterized sigmoid autoencoders
Y. Jiang and C. Pehlevan · 2020
Cited alongside, same era.
Attention approximates sparse distributed memory
T. Bricken and C. Pehlevan · 2021
Cited alongside, same era.
A mathematical framework for transformer circuits
N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, N. DasSarma, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, and C. Olah · 2021
Cited alongside, same era.
Transformer feed-forward layers are key-value memories
M. Geva, R. Schuster, J. Berant, and O. Levy · 2021
Cited alongside, same era.
Local signal adaptivity: Provable feature learning in neural networks beyond kernels
S. Karp, E. Winston, Y. Li, and A. Singh · 2021
Cited alongside, same era.
Scaling language models: Methods, analysis & insights from training gopher
Original
J. W. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, F. Song, J. Aslanides, S. Henderson, R. Ring, S. Young, et al · 2021
Cited alongside, same era.
Linear transformers are secretly fast weight programmers
I. Schlag, K. Irie, and J. Schmidhuber · 2021
Cited alongside, same era.
Approximating how single head attention learns
Original
C. Snell, R. Zhong, D. Klein, and J. Steinhardt · 2021
Cited alongside, same era.
Tensor programs iv: Feature learning in infinite-width neural networks
G. Yang and E. J. Hu · 2021
Cited alongside, same era.
Tensor programs v: Tuning large neural networks via zero-shot hyperparameter transfer
G. Yang, E. J. Hu, I. Babuschkin, S. Sidor, X. Liu, D. Farhi, N. Ryder, J. Pachocki, W. Chen, and J. Gao · 2021
Cited alongside, same era.

Then

Towards understanding grokking: An effective theory of representation learning
Z. Liu, O. Kitouni, N. S. Nolte, E. Michaud, M. Tegmark, and M. Williams · 2022
Later among the works it cites.
Locating and editing factual associations in gpt
K. Meng, D. Bau, A. Andonian, and Y. Belinkov · 2022
Later among the works it cites.
Saturated transformers are constant-depth threshold circuits
W. Merrill, A. Sabharwal, and N. A. Smith · 2022
Later among the works it cites.
Rethinking the role of demonstrations: What makes in-context learning work?
S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer · 2022
Later among the works it cites.
In-context learning and induction heads
C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, S. Johnston, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, and C. Olah · 2022
Later among the works it cites.
Impact of pretraining term frequencies on few-shot reasoning
Y. Razeghi, R. L. Logan IV, M. Gardner, and S. Singh · 2022
Later among the works it cites.
On the effect of pretraining corpora on in-context learning by a large-scale language model
S. Shin, S.-W. Lee, H. Ahn, S. Kim, H. Kim, B. Kim, K. Cho, G. Lee, W. Park, J.-W. Ha, et al · 2022
Later among the works it cites.
An explanation of in-context learning as implicit bayesian inference
S. M. Xie, A. Raghunathan, P. Liang, and T. Ma · 2022
Later among the works it cites.
Unveiling transformers with lego: a synthetic reasoning task
Original
Y. Zhang, A. Backurs, S. Bubeck, R. Eldan, S. Gunasekar, and T. Wagner · 2022
Later among the works it cites.
What learning algorithm is in-context learning? investigations with linear models
E. Akyürek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou · 2023
Closest in time.
Backward feature correction: How deep learning performs deep learning
Z. Allen-Zhu and Y. Li · 2023
Closest in time.
Towards understanding ensemble, knowledge distillation and self-distillation in deep learning
Z. Allen-Zhu and Y. Li · 2023
Closest in time.
Scaling laws for associative memories
Original
V. Cabannes, E. Dohmatob, and A. Bietti · 2023
Closest in time.
Dissecting recall of factual associations in auto-regressive language models
Original
M. Geva, J. Bastings, K. Filippova, and A. Globerson · 2023
Closest in time.
How do transformers learn topic structure: Towards a mechanistic understanding
Y. Li, Y. Li, and A. Risteski · 2023
Closest in time.
Transformers learn shortcuts to automata
B. Liu, J. T. Ash, S. Goel, A. Krishnamurthy, and C. Zhang · 2023
Closest in time.
Progress measures for grokking via mechanistic interpretability
N. Nanda, L. Chan, T. Liberum, J. Smith, and J. Steinhardt · 2023
Closest in time.
Representational strengths and limitations of transformers
C. Sanford, D. Hsu, and M. Telgarsky · 2023
Closest in time.
Scan and snap: Understanding training dynamics and token composition in 1-layer transformer
Y. Tian, Y. Wang, B. Chen, and S. Du · 2023
Closest in time.
Transformers learn in-context by gradient descent
J. Von Oswald, E. Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, A. Zhmoginov, and M. Vladymyrov · 2023
Closest in time.
Interpretability in the wild: a circuit for indirect object identification in gpt-2 small
K. Wang, A. Variengien, A. Conmy, B. Shlegeris, and J. Steinhardt · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…