TempLM: Distilling Language Models into Template-Based Generators

Understand

While pretrained language models (PLMs) have greatly improved text generation, they have also been known to produce unfaithful or inappropriate content.

• In contrast, classic template-based systems provide strong guarantees of faithfulness at the cost of fluency.
• We propose TempLM, which achieves the best of both worlds by distilling a PLM into a template-based generator.
• On the E2E and SynthBio data-to-text datasets, we show that TempLM is more faithful than the original PLM and is more fluent than prior template systems.