Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidance

Understand

Multi-objective learning (MOL) problems often arise in emerging machine learning problems when there are multiple learning criteria, data modalities, or learning tasks.

• Different from single-objective learning, one of the critical challenges in MOL is the potential conflict among different objectives during the iterative optimization process.
• Recent works have developed various dynamic weighting algorithms for MOL such as MGDA and its variants, where the central idea is to find an update direction that avoids conflicts among objectives.
• Albeit its appealing intuition, empirical studies show that dynamic weighting methods may not always outperform static ones.