Unsupervised Transfer Learning for Spoken Language Understanding in Intelligent Agents

Understand

User interaction with voice-powered agents generates large amounts of unlabeled utterances.

• In this paper, we explore techniques to efficiently transfer the knowledge from these unlabeled utterances to improve model performance on Spoken Language Understanding (SLU) tasks.
• We use Embeddings from Language Model (ELMo) to take advantage of unlabeled data by learning contextualized word representations.
• Additionally, we propose ELMo-Light (ELMoL), a faster and simpler unsupervised pre-training method for SLU.