TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents

Understand

We introduce a new approach to generative data-driven dialogue systems (e.g.

• chatbots) called TransferTransfo which is a combination of a Transfer learning based training scheme and a high-capacity Transformer model.
• Fine-tuning is performed by using a multi-task objective which combines several unsupervised prediction tasks.
• The resulting fine-tuned model shows strong improvements over the current state-of-the-art end-to-end conversational models like memory augmented seq2seq and information-retrieval models.