Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning

Understand

Pre-trained sequence-to-sequence language models have led to widespread success in many natural language generation tasks.

• However, there has been relatively less work on analyzing their ability to generate structured outputs such as graphs.
• Unlike natural language, graphs have distinct structural and semantic properties in the context of a downstream NLP task, e.g., generating a graph that is connected and acyclic can be attributed to its structural constraints, while the semantics of a graph can refer to how meaningfully an edge represents the relation between two node concepts.
• In this work, we study pre-trained language models that generate explanation graphs in an end-to-end manner and analyze their ability to learn the structural constraints and semantics of such graphs.