Investigating sanity checks for saliency maps with image and text classification

Understand

Saliency maps have shown to be both useful and misleading for explaining model predictions especially in the context of images.

• In this paper, we perform sanity checks for text modality and show that the conclusions made for image do not directly transfer to text.
• We also analyze the effects of the input multiplier in certain saliency maps using similarity scores, max-sensitivity and infidelity evaluation metrics.
• Our observations reveal that the input multiplier carries input's structural patterns in explanation maps, thus leading to similar results regardless of the choice of model parameters.