Self-Compatibility: Evaluating Causal Discovery without Ground Truth

Understand

As causal ground truth is incredibly rare, causal discovery algorithms are commonly only evaluated on simulated data.

• This is concerning, given that simulations reflect preconceptions about generating processes regarding noise distributions, model classes, and more.
• In this work, we propose a novel method for falsifying the output of a causal discovery algorithm in the absence of ground truth.
• Our key insight is that while statistical learning seeks stability across subsets of data points, causal learning should seek stability across subsets of variables.