InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques

Understand

Mechanistic interpretability methods aim to identify the algorithm a neural network implements, but it is difficult to validate such methods when the true algorithm is unknown.

• This work presents InterpBench, a collection of semi-synthetic yet realistic transformers with known circuits for evaluating these techniques.
• We train simple neural networks using a stricter version of Interchange Intervention Training (IIT) which we call Strict IIT (SIIT).
• Like the original, SIIT trains neural networks by aligning their internal computation with a desired high-level causal model, but it also prevents non-circuit nodes from affecting the model's output.