Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing

Understand

Machine learning (ML) holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities.

• An important step is to characterize the (un)fairness of ML models - their tendency to perform differently across subgroups of the population - and to understand its underlying mechanisms.
• One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data.
• However, diagnosing this phenomenon is difficult, especially when sensitive attributes are causally linked with disease.