LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics

Understand

Learning manipulable representations of the world and its dynamics is central to AI.

• Joint-Embedding Predictive Architectures (JEPAs) offer a promising blueprint, but lack of practical guidance and theory has led to ad-hoc R&D.
• We present a comprehensive theory of JEPAs and instantiate it in {\bf LeJEPA}, a lean, scalable, and theoretically grounded training objective.
• First, we identify the isotropic Gaussian as the optimal distribution that JEPAs' embeddings should follow to minimize downstream prediction risk.