A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders

Understand

Sparse Autoencoders (SAEs) aim to decompose the activation space of large language models (LLMs) into human-interpretable latent directions or features.

• As we increase the number of features in the SAE, hierarchical features tend to split into finer features ("math" may split into "algebra", "geometry", etc.), a phenomenon referred to as feature splitting.
• However, we show that sparse decomposition and splitting of hierarchical features is not robust.
• Specifically, we show that seemingly monosemantic features fail to fire where they should, and instead get "absorbed" into their children features.