Estimation of prediction error with known covariate shift

Understand

In supervised learning, the estimation of prediction error on unlabeled test data is an important task.

• Existing methods are usually built on the assumption that the training and test data are sampled from the same distribution, which is often violated in practice.
• As a result, traditional estimators like cross-validation (CV) will be biased and this may result in poor model selection.
• In this paper, we assume that we have a test dataset in which the feature values are available but not the outcome labels, and focus on a particular form of distributional shift called "covariate shift".