Decision Explanation and Feature Importance for Invertible Networks

Understand

Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature.

• The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model.
• An invertible network classifier can be viewed as a two-stage model: (1) invertible transformation from input space to the feature space; (2) a linear classifier in the feature space.
• We can determine the decision boundary of a linear classifier in the feature space; since the transform is invertible, we can invert the decision boundary from the feature space to the input space.