Analyzing the Generalization and Reliability of Steering Vectors

Understand

Steering vectors (SVs) have been proposed as an effective approach to adjust language model behaviour at inference time by intervening on intermediate model activations.

• They have shown promise in terms of improving both capabilities and model alignment.
• However, the reliability and generalisation properties of this approach are unknown.
• In this work, we rigorously investigate these properties, and show that steering vectors have substantial limitations both in- and out-of-distribution.