Task Loss Estimation for Sequence Prediction

Understand

Often, the performance on a supervised machine learning task is evaluated with a emph{task loss} function that cannot be optimized directly.

• Examples of such loss functions include the classification error, the edit distance and the BLEU score.
• A common workaround for this problem is to instead optimize a emph{surrogate loss} function, such as for instance cross-entropy or hinge loss.
• In order for this remedy to be effective, it is important to ensure that minimization of the surrogate loss results in minimization of the task loss, a condition that we call emph{consistency with the task loss}.