GreaseLM: Graph REASoning Enhanced Language Models for Question Answering

Understand

Answering complex questions about textual narratives requires reasoning over both stated context and the world knowledge that underlies it.

• However, pretrained language models (LM), the foundation of most modern QA systems, do not robustly represent latent relationships between concepts, which is necessary for reasoning.
• While knowledge graphs (KG) are often used to augment LMs with structured representations of world knowledge, it remains an open question how to effectively fuse and reason over the KG representations and the language context, which provides situational constraints and nuances.
• In this work, we propose GreaseLM, a new model that fuses encoded representations from pretrained LMs and graph neural networks over multiple layers of modality interaction operations.