Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection

Understand

The ability to control for the kinds of information encoded in neural representation has a variety of use cases, especially in light of the challenge of interpreting these models.

• We present Iterative Null-space Projection (INLP), a novel method for removing information from neural representations.
• Our method is based on repeated training of linear classifiers that predict a certain property we aim to remove, followed by projection of the representations on their null-space.
• By doing so, the classifiers become oblivious to that target property, making it hard to linearly separate the data according to it.