Unsupervised Paraphrasing with Pretrained Language Models

Understand

Paraphrase generation has benefited extensively from recent progress in the designing of training objectives and model architectures.

• However, previous explorations have largely focused on supervised methods, which require a large amount of labeled data that is costly to collect.
• To address this drawback, we adopt a transfer learning approach and propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting.
• Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking (DB).