Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions

Understand

Understanding the inner workings of large language models (LLMs) is crucial for advancing their theoretical foundations and real-world applications.

• While the attention mechanism and multi-layer perceptrons (MLPs) have been studied independently, their interactions remain largely unexplored.
• This study investigates how attention heads and next-token neurons interact in LLMs to predict new words.
• We propose a methodology to identify next-token neurons, find prompts that highly activate them, and determine the upstream attention heads responsible.