Interpretable and Testable Vision Features via Sparse Autoencoders

Understand

To truly understand vision models, we must not only interpret their learned features but also validate these interpretations through controlled experiments.

• While earlier work offers either rich semantics or direct control, few post-hoc tools supply both in a single, model-agnostic procedure.
• We use sparse autoencoders (SAEs) to bridge this gap; each sparse feature comes with real-image exemplars that reveal its meaning and a decoding vector that can be manipulated to probe its influence on downstream task behavior.
• By applying our method to widely-used pre-trained vision models, we reveal meaningful differences in the semantic abstractions learned by different pre-training objectives.