A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks

Understand

Much effort has been devoted to evaluate whether multi-task learning can be leveraged to learn rich representations that can be used in various Natural Language Processing (NLP) down-stream applications.

• However, there is still a lack of understanding of the settings in which multi-task learning has a significant effect.
• In this work, we introduce a hierarchical model trained in a multi-task learning setup on a set of carefully selected semantic tasks.
• The model is trained in a hierarchical fashion to introduce an inductive bias by supervising a set of low level tasks at the bottom layers of the model and more complex tasks at the top layers of the model.