Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks

Understand

As Deep Neural Networks (DNNs) have demonstrated superhuman performance in a variety of fields, there is an increasing interest in understanding the complex internal mechanisms of DNNs.

• In this paper, we propose Relative Attributing Propagation (RAP), which decomposes the output predictions of DNNs with a new perspective of separating the relevant (positive) and irrelevant (negative) attributions according to the relative influence between the layers.
• The relevance of each neuron is identified with respect to its degree of contribution, separated into positive and negative, while preserving the conservation rule.
• Considering the relevance assigned to neurons in terms of relative priority, RAP allows each neuron to be assigned with a bi-polar importance score concerning the output: from highly relevant to highly irrelevant.