What went wrong and when? Instance-wise Feature Importance for Time-series Models

Understand

Explanations of time series models are useful for high stakes applications like healthcare but have received little attention in machine learning literature.

• We propose FIT, a framework that evaluates the importance of observations for a multivariate time-series black-box model by quantifying the shift in the predictive distribution over time.
• FIT defines the importance of an observation based on its contribution to the distributional shift under a KL-divergence that contrasts the predictive distribution against a counterfactual where the rest of the features are unobserved.
• We also demonstrate the need to control for time-dependent distribution shifts.