Conditional Variance Penalties and Domain Shift Robustness

Understand

When training a deep neural network for image classification, one can broadly distinguish between two types of latent features of images that will drive the classification.

• We can divide latent features into (i) "core" or "conditionally invariant" features $X^\text{core}$ whose distribution $X^\text{core}\vert Y$, conditional on the class $Y$, does not change substantially across domains and (ii) "style" features $X^{\text{style}}$ whose distribution $X^{\text{style}} \vert Y$ can change substantially across domains.
• Examples for style features include position, rotation, image quality or brightness but also more complex ones like hair color, image quality or posture for images of persons.
• Our goal is to minimize a loss that is robust under changes in the distribution of these style features.