Minimax Optimal Estimation of Stability Under Distribution Shift

Understand

The performance of decision policies and prediction models often deteriorates when applied to environments different from the ones seen during training.

• To ensure reliable operation, we analyze the stability of a system under distribution shift, which is defined as the smallest change in the underlying environment that causes the system's performance to deteriorate beyond a permissible threshold.
• In contrast to standard tail risk measures and distributionally robust losses that require the specification of a plausible magnitude of distribution shift, the stability measure is defined in terms of a more intuitive quantity: the level of acceptable performance degradation.
• We develop a minimax optimal estimator of stability and analyze its convergence rate, which exhibits a fundamental phase shift behavior.