Eliciting Latent Predictions from Transformers with the Tuned Lens

Understand

We analyze transformers from the perspective of iterative inference, seeking to understand how model predictions are refined layer by layer.

• To do so, we train an affine probe for each block in a frozen pretrained model, making it possible to decode every hidden state into a distribution over the vocabulary.
• Our method, the tuned lens, is a refinement of the earlier "logit lens" technique, which yielded useful insights but is often brittle.
• We test our method on various autoregressive language models with up to 20B parameters, showing it to be more predictive, reliable and unbiased than the logit lens.