Exploring Shared Structures and Hierarchies for Multiple NLP Tasks

Understand

Designing shared neural architecture plays an important role in multi-task learning.

• The challenge is that finding an optimal sharing scheme heavily relies on the expert knowledge and is not scalable to a large number of diverse tasks.
• Inspired by the promising work of neural architecture search (NAS), we apply reinforcement learning to automatically find possible shared architecture for multi-task learning.
• Specifically, we use a controller to select from a set of shareable modules and assemble a task-specific architecture, and repeat the same procedure for other tasks.