Pretraining Methods for Dialog Context Representation Learning

Understand

This paper examines various unsupervised pretraining objectives for learning dialog context representations.

• Two novel methods of pretraining dialog context encoders are proposed, and a total of four methods are examined.
• Each pretraining objective is fine-tuned and evaluated on a set of downstream dialog tasks using the MultiWoz dataset and strong performance improvement is observed.
• Further evaluation shows that our pretraining objectives result in not only better performance, but also better convergence, models that are less data hungry and have better domain generalizability.