Metric-Optimized Example Weights

Understand

Real-world machine learning applications often have complex test metrics, and may have training and test data that are not identically distributed.

• Motivated by known connections between complex test metrics and cost-weighted learning, we propose addressing these issues by using a weighted loss function with a standard loss, where the weights on the training examples are learned to optimize the test metric on a validation set.
• These metric-optimized example weights can be learned for any test metric, including black box and customized ones for specific applications.
• We illustrate the performance of the proposed method on diverse public benchmark datasets and real-world applications.