ViT-CX: Causal Explanation of Vision Transformers

Understand

Despite the popularity of Vision Transformers (ViTs) and eXplainable AI (XAI), only a few explanation methods have been designed specially for ViTs thus far.

• They mostly use attention weights of the [CLS] token on patch embeddings and often produce unsatisfactory saliency maps.
• This paper proposes a novel method for explaining ViTs called ViT-CX.
• It is based on patch embeddings, rather than attentions paid to them, and their causal impacts on the model output.