Sparse Autoencoders Find Highly Interpretable Features in Language Models

Understand

One of the roadblocks to a better understanding of neural networks' internals is \textit{polysemanticity}, where neurons appear to activate in multiple, semantically distinct contexts.

• Polysemanticity prevents us from identifying concise, human-understandable explanations for what neural networks are doing internally.
• One hypothesised cause of polysemanticity is \textit{superposition}, where neural networks represent more features than they have neurons by assigning features to an overcomplete set of directions in activation space, rather than to individual neurons.
• Here, we attempt to identify those directions, using sparse autoencoders to reconstruct the internal activations of a language model.