Efficiently Tuned Parameters are Task Embeddings

Understand

Intermediate-task transfer can benefit a wide range of NLP tasks with properly selected source datasets.

• However, it is computationally infeasible to experiment with all intermediate transfer combinations, making choosing a useful source task a challenging problem.
• In this paper, we anticipate that task-specific parameters updated in parameter-efficient tuning methods are likely to encode task-specific information.
• Therefore, such parameters can be predictive for inter-task transferability.