Supervised Learning: No Loss No Cry

Understand

Supervised learning requires the specification of a loss function to minimise.

• While the theory of admissible losses from both a computational and statistical perspective is well-developed, these offer a panoply of different choices.
• In practice, this choice is typically made in an \emph{ad hoc} manner.
• In hopes of making this procedure more principled, the problem of \emph{learning the loss function} for a downstream task (e.g., classification) has garnered recent interest.