How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding

Training Verifiers to Solve Math Word Problems

Original

Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J. 2021 · 2021

Earlier work this paper cites.

Causal abstractions of neural networks

Geiger, A.; Lu, H.; Icard, T.; and Potts, C. 2021 · 2021

Earlier work this paper cites.

Probing classifiers: Promises, shortcomings, and advances

Belinkov, Y. 2022 · 2022

Earlier work this paper cites.

Toy models of superposition

Original

Elhage, N.; Hume, T.; Olsson, C.; Schiefer, N.; Henighan, T.; Kravec, S.; Hatfield-Dodds, Z.; Lasenby, R.; Drain, D.; Chen, C.; et al. 2022 · 2022

Earlier work this paper cites.

Large language models are zero-shot reasoners

Kojima, T.; Gu, S. S.; Reid, M.; Matsuo, Y.; and Iwasawa, Y. 2022 · 2022

Earlier work this paper cites.

Locating and editing factual associations in gpt

Meng, K.; Bau, D.; Andonian, A.; and Belinkov, Y. 2022 · 2022

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022 · 2022

Earlier work this paper cites.

Faithfulness tests for natural language explanations

Original

Atanasova, P.; Camburu, O.-M.; Lioma, C.; Lukasiewicz, T.; Simonsen, J. G.; and Augenstein, I. 2023 · 2023

Earlier work this paper cites.

Language models can explain neurons in language models

Bills, S.; Cammarata, N.; Mossing, D.; Tillman, H.; Gao, L.; Goh, G.; Sutskever, I.; Leike, J.; Wu, J.; and Saunders, W. 2023 · 2023

Earlier work this paper cites.

Towards monosemanticity: Decomposing language models with dictionary learning

Bricken, T.; Templeton, A.; Batson, J.; Chen, B.; Jermyn, A.; Conerly, T.; Turner, N.; Anil, C.; Denison, C.; Askell, A.; et al. 2023 · 2023

Earlier work this paper cites.

Sparse autoencoders find highly interpretable features in language models

Original

Cunningham, H.; Ewart, A.; Riggs, L.; Huben, R.; and Sharkey, L. 2023 · 2023

Earlier work this paper cites.

Causal abstraction: A theoretical foundation for mechanistic interpretability

Original

Geiger, A.; Ibeling, D.; Zur, A.; Chaudhary, M.; Chauhan, S.; Huang, J.; Arora, A.; Wu, Z.; Goodman, N.; Potts, C.; et al. 2023 · 2023

Earlier work this paper cites.

Localizing model behavior with path patching

Original

Goldowsky-Dill, N.; MacLeod, C.; Sato, L.; and Arora, A. 2023 · 2023

Earlier work this paper cites.

How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

Hanna, M.; Liu, O.; and Variengien, A. 2023 · 2023

Earlier work this paper cites.

Is this the subspace you are looking for? an interpretability illusion for subspace activation patching

Original

Makelov, A.; Lange, G.; and Nanda, N. 2023 · 2023

Cited alongside, same era.

Progress measures for grokking via mechanistic interpretability, 2023

Original

Nanda, N.; Chan, L.; Lieberum, T.; Smith, J.; and Steinhardt, J. 2023 · 2023

Cited alongside, same era.

A reply to makelov et al.(2023)’s” interpretability illusion” arguments

Original

Wu, Z.; Geiger, A.; Huang, J.; Arora, A.; Icard, T.; Potts, C.; and Goodman, N. D. 2024 · 2023

Cited alongside, same era.

Towards best practices of activation patching in language models: Metrics and methods

Original

Zhang, F.; and Nanda, N. 2023 · 2023

Cited alongside, same era.

Faithfulness vs. plausibility: On the (un) reliability of explanations from large language models

Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approach

Original

Li, Q.; Li, J.; Liu, T.; Zeng, Y.; Cheng, M.; Huang, W.; and Liu, Q. 2024 · 2024

Later among the works it cites.

Sparse feature circuits: Discovering and editing interpretable causal graphs in language models

Original

Marks, S.; Rager, C.; Michaud, E. J.; Belinkov, Y.; Bau, D.; and Mueller, A. 2024 · 2024

Later among the works it cites.

Analyzing (In) Abilities of SAEs via Formal Languages

Original

Menon, A.; Shrivastava, M.; Krueger, D.; and Lubana, E. S. 2024 · 2024

Later among the works it cites.

Making reasoning matter: Measuring and improving faithfulness of chain-of-thought reasoning

Original

Paul, D.; West, R.; Bosselut, A.; and Faltings, B. 2024 · 2024

Later among the works it cites.