Multi-Layer Attention-Based Explainability via Transformers for Tabular Data

Understand

We propose a graph-oriented attention-based explainability method for tabular data.

• Tasks involving tabular data have been solved mostly using traditional tree-based machine learning models which have the challenges of feature selection and engineering.
• With that in mind, we consider a transformer architecture for tabular data, which is amenable to explainability, and present a novel way to leverage self-attention mechanism to provide explanations by taking into account the attention matrices of all heads and layers as a whole.
• The matrices are mapped to a graph structure where groups of features correspond to nodes and attention values to arcs.