Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictions

Understand

Weak-to-Strong Generalization (W2SG), where a weak model supervises a stronger one, serves as an important analogy for understanding how humans might guide superhuman intelligence in the future.

• Promising empirical results revealed that a strong model can surpass its weak supervisor.
• While recent work has offered theoretical insights into this phenomenon, a clear understanding of the interactions between weak and strong models that drive W2SG remains elusive.
• We investigate W2SG through a theoretical lens and show that it can be characterized using kernels derived from the principal components of weak and strong models' internal representations.