Entities as Experts: Sparse Memory Access with Entity Supervision

Understand

We focus on the problem of capturing declarative knowledge about entities in the learned parameters of a language model.

• We introduce a new model - Entities as Experts (EAE) - that can access distinct memories of the entities mentioned in a piece of text.
• Unlike previous efforts to integrate entity knowledge into sequence models, EAE's entity representations are learned directly from text.
• We show that EAE's learned representations capture sufficient knowledge to answer TriviaQA questions such as "Which Dr.