Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning

Understand

A family of loss functions built on pair-based computation have been proposed in the literature which provide a myriad of solutions for deep metric learning.

• In this paper, we provide a general weighting framework for understanding recent pair-based loss functions.
• Our contributions are three-fold: (1) we establish a General Pair Weighting (GPW) framework, which casts the sampling problem of deep metric learning into a unified view of pair weighting through gradient analysis, providing a powerful tool for understanding recent pair-based loss functions; (2) we show that with GPW, various existing pair-based methods can be compared and discussed comprehensively, with clear differences and key limitations identified; (3) we propose a new loss called multi-similarity loss (MS loss) under the GPW, which is implemented in two iterative steps (i.e., mining and weighting).
• This allows it to fully consider three similarities for pair weighting, providing a more principled approach for collecting and weighting informative pairs.