Empowering Language Models with Knowledge Graph Reasoning for Question Answering

Understand

Answering open-domain questions requires world knowledge about in-context entities.

• As pre-trained Language Models (LMs) lack the power to store all required knowledge, external knowledge sources, such as knowledge graphs, are often used to augment LMs.
• In this work, we propose knOwledge REasOning empowered Language Model (OREO-LM), which consists of a novel Knowledge Interaction Layer that can be flexibly plugged into existing Transformer-based LMs to interact with a differentiable Knowledge Graph Reasoning module collaboratively.
• In this way, LM guides KG to walk towards the desired answer, while the retrieved knowledge improves LM.