Auditing Black-box Models for Indirect Influence

Understand

Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score.

• It is therefore hard to acquire a deeper understanding of model behavior, and in particular how different features influence the model prediction.
• This is important when interpreting the behavior of complex models, or asserting that certain problematic attributes (like race or gender) are not unduly influencing decisions.
• In this paper, we present a technique for auditing black-box models, which lets us study the extent to which existing models take advantage of particular features in the dataset, without knowing how the models work.