Imbalance Trouble: Revisiting Neural-Collapse Geometry

Understand

Neural Collapse refers to the remarkable structural properties characterizing the geometry of class embeddings and classifier weights, found by deep nets when trained beyond zero training error.

• However, this characterization only holds for balanced data.
• Here we thus ask whether it can be made invariant to class imbalances.
• Towards this end, we adopt the unconstrained-features model (UFM), a recent theoretical model for studying neural collapse, and introduce Simplex-Encoded-Labels Interpolation (SELI) as an invariant characterization of the neural collapse phenomenon.