Auto-Lambda: Disentangling Dynamic Task Relationships

Understand

Understanding the structure of multiple related tasks allows for multi-task learning to improve the generalisation ability of one or all of them.

• However, it usually requires training each pairwise combination of tasks together in order to capture task relationships, at an extremely high computational cost.
• In this work, we learn task relationships via an automated weighting framework, named Auto-Lambda.
• Unlike previous methods where task relationships are assumed to be fixed, Auto-Lambda is a gradient-based meta learning framework which explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formulation of a meta-loss; where the validation loss automatically influences task weightings throughout training.