Transcoders Find Interpretable LLM Feature Circuits

Correlating neural and symbolic representations of language

Original

Chrupała, G. and Alishahi, A · 1905

Earlier work this paper cites.

Language Models are Few-Shot Learners, July 2020

Original

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D · 2005

Earlier work this paper cites.

The mythos of model interpretability, 2017

Lipton, Z. C · 2017

Earlier work this paper cites.

Feature visualization

Olah, C., Mordvintsev, A., and Schubert, L · 2017

Earlier work this paper cites.

OpenWebText Corpus, 2019

Gokaslan, A. and Cohen, V · 2019

Earlier work this paper cites.

Language Models are Unsupervised Multitask Learners

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I · 2019

Earlier work this paper cites.

Zoom In: An Introduction to Circuits

Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., and Carter, S · 2020

Earlier work this paper cites.

Investigating Gender Bias in Language Models Using Causal Mediation Analysis

Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Singer, Y., and Shieber, S · 2020

Earlier work this paper cites.

An interpretability illusion for bert

Original

Bolukbasi, T., Pearce, A., Yuan, A., Coenen, A., Reif, E., Viégas, F., and Wattenberg, M · 2021

Earlier work this paper cites.

A Mathematical Framework for Transformer Circuits

Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., DasSarma, N., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., and Olah, C · 2021

Earlier work this paper cites.

Causal Abstractions of Neural Networks, October 2021

Original

Geiger, A., Lu, H., Icard, T., and Potts, C · 2021

Earlier work this paper cites.

Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases, 2022

Chris Olah · 2022

Earlier work this paper cites.

Softmax linear units

Elhage, N., Hume, T., Olsson, C., Nanda, N., Henighan, T., Johnston, S., ElShowk, S., Joseph, N., DasSarma, N., Mann, B., Hernandez, D., Askell, A., Ndousse, K., Jones, A., Drain, D., Chen, A., Bai, Y., Ganguli, D., Lovitt, L., Hatfield-Dodds, Z., Kernion, J., Conerly, T., Kravec, S., Fort, S., Kadavath, S., Jacobson, J., Tran-Johnson, E., Kaplan, J., Clark, J., Brown, T., McCandlish, S., Amodei, D., and Olah, C · 2022

Earlier work this paper cites.

TransformerLens, 2022

Nanda, N. and Bloom, J · 2022

Earlier work this paper cites.

In-context Learning and Induction Heads, September 2022

Original

Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., and Olah, C · 2022

Earlier work this paper cites.

Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small, November 2022

Original

Wang, K., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J · 2022

Earlier work this paper cites.

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling, May 2023

Original

Biderman, S., Schoelkopf, H., Anthony, Q., Bradley, H., O’Brien, K., Hallahan, E., Khan, M. A., Purohit, S., Prashanth, U. S., Raff, E., Skowron, A., Sutawika, L., and van der Wal, O · 2023

Cited alongside, same era.

Language models can explain neurons in language models, 2023

Bills, S., Cammarata, N., Mossing, D., Tillman, H., Gao, L., Goh, G., Sutskever, I., Leike, J., Wu, J., and Saunders, W · 2023

Cited alongside, same era.

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J. E., Hume, T., Carter, S., Henighan, T., and Olah, C · 2023

Cited alongside, same era.

Towards Automated Circuit Discovery for Mechanistic Interpretability, October 2023

Original

Conmy, A., Mavor-Parker, A. N., Lynch, A., Heimersheim, S., and Garriga-Alonso, A · 2023

Cited alongside, same era.

Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors, 2023

Yun, Z., Chen, Y., Olshausen, B. A., and LeCun, Y · 2023

Later among the works it cites.

Using features for easy circuit identification, 2024

Batson, J., Chen, B., and Jones, A · 2024

Closest in time.

Case Studies in Reverse-Engineering Sparse Autoencoder Features by Using MLP Linearization, 2024

Dunefsky, J., Chlenski, P., Rajamanoharan, S., and Nanda, N · 2024

Closest in time.

A Primer on the Inner Workings of Transformer-based Language Models, May 2024

Original

Ferrando, J., Sarti, G., Bisazza, A., and Costa-jussà, M. R · 2024

Closest in time.

Interpreting CLIP’s Image Representation via Text-Based Decomposition, March 2024

Original

Gandelsman, Y., Efros, A. A., and Steinhardt, J · 2024

Closest in time.