pyvene: A Library for Understanding and Improving PyTorch Models via Interventions

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack

Original

Zhezhi He, Adnan Siraj Rakin, and Deliang Fan. 2019 · 2019

Earlier work this paper cites.

A structural probe for finding syntax in word representations

John Hewitt and Christopher D Manning. 2019 · 2019

Earlier work this paper cites.

Amnesic probing: Behavioral explanation with amnesic counterfactuals

Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg. 2020 · 2020

Earlier work this paper cites.

Neural natural language inference models partially embed theories of lexical entailment and negation

Atticus Geiger, Kyle Richardson, and Christopher Potts. 2020 · 2020

Earlier work this paper cites.

Investigating gender bias in language models using causal mediation analysis

Original

Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020 · 2020

Earlier work this paper cites.

Causal abstractions of neural networks

Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. 2021 · 2021

Earlier work this paper cites.

Causal scrubbing: a method for rigorously testing interpretability hypotheses

Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. 2022 · 2022

Earlier work this paper cites.

Inducing causal structure for interpretable neural networks

Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah Goodman, and Christopher Potts. 2022 · 2022

Cited alongside, same era.

Locating and editing factual associations in GPT

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022 · 2022

Cited alongside, same era.

Transformerlens

Neel Nanda and Joseph Bloom. 2022 · 2022

Cited alongside, same era.

Pythia: A suite for analyzing large language models across training and scaling

Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar Van Der Wal. 2023 · 2023

Cited alongside, same era.

Towards automated circuit discovery for mechanistic interpretability

Original

Arthur Conmy, Augustine N Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. 2023 · 2023

Cited alongside, same era.

Localizing model behavior with path patching

Original

Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. 2023 · 2023

Later among the works it cites.

Backpack language models

John Hewitt, John Thickstun, Christopher Manning, and Percy Liang. 2023 · 2023

Later among the works it cites.

Uncovering intermediate variables in transformers using circuit probing

Original

Michael A Lepori, Thomas Serre, and Ellie Pavlick. 2023 · 2023

Later among the works it cites.

graphpatch

Evan Lloyd. 2023 · 2023

Later among the works it cites.

Interpretability in the wild: A circuit for indirect object identification in GPT-2 small

Original

Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023 · 2023

Later among the works it cites.