Parameter-efficient Multi-Task and Multi-Domain Learning using Factorized Tensor Networks

Understand

Multi-task and multi-domain learning methods seek to learn multiple tasks/domains, jointly or one after another, using a single unified network.

• The primary challenge and opportunity lie in leveraging shared information across these tasks and domains to enhance the efficiency of the unified network.
• The efficiency can be in terms of accuracy, storage cost, computation, or sample complexity.
• In this paper, we introduce a factorized tensor network (FTN) designed to achieve accuracy comparable to that of independent single-task or single-domain networks, while introducing a minimal number of additional parameters.