Interpreting Multivariate Shapley Interactions in DNNs

Understand

This paper aims to explain deep neural networks (DNNs) from the perspective of multivariate interactions.

• In this paper, we define and quantify the significance of interactions among multiple input variables of the DNN.
• Input variables with strong interactions usually form a coalition and reflect prototype features, which are memorized and used by the DNN for inference.
• We define the significance of interactions based on the Shapley value, which is designed to assign the attribution value of each input variable to the inference.