Built on

Understanding intermediate layers using linear classifier probes
Original
Guillaume Alain and Yoshua Bengio · 2016
Earlier work this paper cites.
Attention is all you need
A Vaswani · 2017
Earlier work this paper cites.
A structural probe for finding syntax in word representations
John Hewitt and Christopher D Manning · 2019
Earlier work this paper cites.
Emergent linguistic structure in artificial neural networks trained by self-supervision
Christopher D Manning, Kevin Clark, John Hewitt, et al · 2020
Earlier work this paper cites.
A mathematical framework for transformer circuits
Nelson Elhage, Neel Nanda, Catherine Olsson, et al · 2021
Earlier work this paper cites.
Toy models of superposition
Original
Nelson Elhage, Tristan Hume, Catherine Olsson, et al · 2022
Earlier work this paper cites.
Emergent world representations: Exploring a sequence model trained on a synthetic task
Original
Kenneth Li, Aspen K Hopkins, David Bau, et al · 2022
Earlier work this paper cites.
Acquisition of chess knowledge in alphazero
Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, et al · 2022
Earlier work this paper cites.
Transformerlens
Neel Nanda and Joseph Bloom · 2022
Earlier work this paper cites.
In-context Learning and Induction Hea ds
Original
Catherine Olsson, Nelson Elhage, Neel Nanda, et al · 2022
Earlier work this paper cites.

Similar

Sparse Relational Reasoning with Object-Centric Representations, July 2022
Original
Alex F. Spies, Alessandra Russo, and Murray Shanahan · 2022
Cited alongside, same era.
Eliciting Latent Predictions from Transformers with the Tuned Lens
Original
Nora Belrose, Zach Furman, Logan Smith, et al · 2023
Cited alongside, same era.
Towards monosemanticity: Decomposing language models with dictionary learning
Trenton Bricken, Adly Templeton, Brian Chen, et al · 2023
Cited alongside, same era.
Sparse Autoencoders Find Highly Interpretable Features in Language Models, October 2023
Original
Hoagy Cunningham, Aidan Ewart, Logan Riggs, et al · 2023
Cited alongside, same era.
A configurable library for generating and manipulating maze datasets
Original

Then

Dictionary learning improves patch-free circuit discovery in mechanistic interpretability: A case study on othello-gpt
Original
Zhengfu He, Xuyang Ge, Qiong Tang, et al · 2024
Closest in time.
Linearly structured world representations in maze-solving transformers
Michael I. Ivanitskiy, Alex F. Spies, Tilman Räuker, et al · 2024
Closest in time.
Evidence of learned look-ahead in a chess-playing neural network
Original
Erik Jenner, Shreyas Kapur, Vasil Georgiev, et al · 2024
Closest in time.
Anthropic circuits Updates - January 2024, January 2024
Adam Jermyn and Adly Templeton · 2024
Closest in time.
Emergent world models and latent variable estimation in chess-playing language models
Original
Adam Karvonen · 2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Transformers Use Causal World Models in Maze-Solving Tasks

Spies, Alex F., Edwards, William, Ivanitskiy, Michael I. et al.

Understand

Recent studies in interpretability have explored the inner workings of transformer models trained on tasks across various domains, often discovering that these networks naturally develop highly structured representations.

When such representations comprehensively reflect the task domain's structure, they are commonly referred to as "World Models" (WMs).
In this work, we identify WMs in transformers trained on maze-solving tasks.
By using Sparse Autoencoders (SAEs) and analyzing attention patterns, we examine the construction of WMs and demonstrate consistency between SAE feature-based and circuit-based analyses.

Built on

Understanding intermediate layers using linear classifier probes
Original
Guillaume Alain and Yoshua Bengio · 2016
Earlier work this paper cites.
Attention is all you need
A Vaswani · 2017
Earlier work this paper cites.
A structural probe for finding syntax in word representations
John Hewitt and Christopher D Manning · 2019
Earlier work this paper cites.
Emergent linguistic structure in artificial neural networks trained by self-supervision
Christopher D Manning, Kevin Clark, John Hewitt, et al · 2020
Earlier work this paper cites.
A mathematical framework for transformer circuits
Nelson Elhage, Neel Nanda, Catherine Olsson, et al · 2021
Earlier work this paper cites.
Toy models of superposition
Original
Nelson Elhage, Tristan Hume, Catherine Olsson, et al · 2022
Earlier work this paper cites.
Emergent world representations: Exploring a sequence model trained on a synthetic task
Original
Kenneth Li, Aspen K Hopkins, David Bau, et al · 2022
Earlier work this paper cites.
Acquisition of chess knowledge in alphazero
Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, et al · 2022
Earlier work this paper cites.
Transformerlens
Neel Nanda and Joseph Bloom · 2022
Earlier work this paper cites.
In-context Learning and Induction Hea ds
Original
Catherine Olsson, Nelson Elhage, Neel Nanda, et al · 2022
Earlier work this paper cites.

Similar

Sparse Relational Reasoning with Object-Centric Representations, July 2022
Original
Alex F. Spies, Alessandra Russo, and Murray Shanahan · 2022
Cited alongside, same era.
Eliciting Latent Predictions from Transformers with the Tuned Lens
Original
Nora Belrose, Zach Furman, Logan Smith, et al · 2023
Cited alongside, same era.
Towards monosemanticity: Decomposing language models with dictionary learning
Trenton Bricken, Adly Templeton, Brian Chen, et al · 2023
Cited alongside, same era.
Sparse Autoencoders Find Highly Interpretable Features in Language Models, October 2023
Original
Hoagy Cunningham, Aidan Ewart, Logan Riggs, et al · 2023
Cited alongside, same era.
A configurable library for generating and manipulating maze datasets
Original

Then

Dictionary learning improves patch-free circuit discovery in mechanistic interpretability: A case study on othello-gpt
Original
Zhengfu He, Xuyang Ge, Qiong Tang, et al · 2024
Closest in time.
Linearly structured world representations in maze-solving transformers
Michael I. Ivanitskiy, Alex F. Spies, Tilman Räuker, et al · 2024
Closest in time.
Evidence of learned look-ahead in a chess-playing neural network
Original
Erik Jenner, Shreyas Kapur, Vasil Georgiev, et al · 2024
Closest in time.
Anthropic circuits Updates - January 2024, January 2024
Adam Jermyn and Adly Templeton · 2024
Closest in time.
Emergent world models and latent variable estimation in chess-playing language models
Original
Adam Karvonen · 2024
Closest in time.