Evaluating Synthetic Activations composed of SAE Latents in GPT-2

Understand

Sparse Auto-Encoders (SAEs) are commonly employed in mechanistic interpretability to decompose the residual stream into monosemantic SAE latents.

• Recent work demonstrates that perturbing a model's activations at an early layer results in a step-function-like change in the model's final layer activations.
• Furthermore, the model's sensitivity to this perturbation differs between model-generated (real) activations and random activations.
• In our study, we assess model sensitivity in order to compare real activations to synthetic activations composed of SAE latents.