Synthetic Benchmarks for Scientific Research in Explainable Machine Learning

Understand

As machine learning models grow more complex and their applications become more high-stakes, tools for explaining model predictions have become increasingly important.

• This has spurred a flurry of research in model explainability and has given rise to feature attribution methods such as LIME and SHAP.
• Despite their widespread use, evaluating and comparing different feature attribution methods remains challenging: evaluations ideally require human studies, and empirical evaluation metrics are often data-intensive or computationally prohibitive on real-world datasets.
• In this work, we address this issue by releasing XAI-Bench: a suite of synthetic datasets along with a library for benchmarking feature attribution algorithms.